IP Library › Granted Patent US 12,430,922
Granted Patent B2
US 12,430,922 · App. 17/379,328 · Granted Sep 30, 2025

Device and method for ascertaining a physical property of a physical object

Inventors: Alexander Kugele (Renningen, DE); Michael Pfeiffer (Schoenaich, DE); Thomas Pfeil (Renningen, DE)
Assignee: ROBERT BOSCH GMBH
G06V20/56B60W30/18109G06F18/25G06N3/049G06N3/08G06V10/751
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,430,922
App. No.
17/379,328
Granted
Sep 30, 2025
Kind
B2
Abstract

A method for ascertaining a physical property of an object. The method includes detecting, for each input point in time of a sequence of input points in time, sensor data including information about a physical object, using an event-based sensor; for each subsequence of a breakdown of the sequence of input points in time into multiple subsequences including: feeding the sensor data detected for the input points in time of the subsequence to a pulsed neural network which generates a first processing result of the subsequence; feeding the processing result of the subsequence to a non-pulsed neural network; and processing the processing result of the subsequence by non-pulsed neurons of one or multiple first layer(s) of the non-pulsed neural network for generating a second processing result of the subsequence; and feeding the second processing results of the multiple subsequences to one or multiple second layer(s) of the non-pulsed neural network.

Claims (48)

1. A method for ascertaining a physical property of a physical object, comprising the following steps:

detecting sensor data using an event-based sensor for each input point in time of a sequence of input points in time, the sensor data including pieces of information about the physical object;

for each subsequence of a breakdown of the sequence of input points in time into multiple subsequences:

feeding the sensor data detected for the input points in time of the subsequence to a pulsed neural network,

processing the sensor data of the input points in time of the subsequence by pulsed neurons of the pulsed neural network for generating a first processing result of the subsequence, the pulsed neurons in each case integrating values of sensor data of different input points in time of the subsequence;

feeding the first processing result of the subsequence to a non-pulsed neural network, and

processing the first processing result of the subsequence by non-pulsed neurons of one or multiple first layer(s) of the non-pulsed neural network for generating a second processing result of the subsequence;

feeding the second processing results of the multiple subsequences to one or multiple second layer(s) of the non-pulsed neural network;

combining the second processing results of the multiple subsequences by non-pulsed neurons of the one or multiple second layer(s) of the non-pulsed neural network in that the non-pulsed neurons of the one or multiple second layer(s) in each case calculate a weighted sum of values of second processing results of different subsequences; and

ascertaining the physical property of the physical object from outputs of the non-pulsed neurons of the one or multiple second layer(s), wherein a trigger determines when the processing by the non-pulsed neural network is executed, wherein the trigger corresponds to the pulsed neural network generating a number of pulses that meets a threshold amount of pulses.

2. The method as recited in claim 1 , wherein the event-based sensor supplies sensor data for multiple components of a sensor data vector, and the non-pulsed neurons of the non-pulsed neural network, for generating the second processing result of the subsequence, calculate weighted sums of values of second processing results of different components of the sensor data vector.

3. The method as recited in claim 1 , wherein the input points in time are points in time of events to which the event-based sensor responds by outputting sensor data.

4. The method as recited in claim 1 , further comprising:

feeding, for each subsequence, at least a portion of outputs of the pulsed neurons to a further neural network, the further neural network carrying out a classification which establishes an end of the subsequence.

5. The method as recited in claim 1 , further comprising:

ascertaining, for each subsequence, a number of at least a portion of outputs of the pulsed neurons per unit of time, and ending the subsequence when the outputs of the pulsed neurons per unit of time exceed a threshold value.

6. A method for controlling an actuator, comprising the following steps:

ascertaining a physical property of a physical object by:

detecting sensor data using an event-based sensor for each input point in time of a sequence of input points in time, the sensor data including pieces of information about the physical object;

for each subsequence of a breakdown of the sequence of input points in time into multiple subsequences:

feeding the sensor data detected for the input points in time of the subsequence to a pulsed neural network,

processing the sensor data of the input points in time of the subsequence by pulsed neurons of the pulsed neural network for generating a first processing result of the subsequence, the pulsed neurons in each case integrating values of sensor data of different input points in time of the subsequence;

feeding the first processing result of the subsequence to a non-pulsed neural network, and

processing the first processing result of the subsequence by non-pulsed neurons of one or multiple first layer(s) of the non-pulsed neural network for generating a second processing result of the subsequence;

feeding the second processing results of the multiple subsequences to one or multiple second layer(s) of the non-pulsed neural network;

combining the second processing results of the multiple subsequences by non-pulsed neurons of the one or multiple second layer(s) of the non-pulsed neural network in that the non-pulsed neurons of the one or multiple second layer(s) in each case calculate a weighted sum of values of second processing results of different subsequences; and

ascertaining the physical property of the physical object from outputs of the non-pulsed neurons of the one or multiple second layer(s); and

controlling an actuator as a function of the ascertained physical property of the physical object, wherein a trigger determines when the processing by the non-pulsed neural network is executed, wherein the trigger corresponds to the pulsed neural network generating a number of pulses that meets a threshold amount of pulses.

7. A device configured to ascertain a physical property of a physical object, the device configured to:

detect sensor data using an event-based sensor for each input point in time of a sequence of input points in time, the sensor data including pieces of information about the physical object;

for each subsequence of a breakdown of the sequence of input points in time into multiple subsequences:

feed the sensor data detected for the input points in time of the subsequence to a pulsed neural network,

process the sensor data of the input points in time of the subsequence by pulsed neurons of the pulsed neural network for generating a first processing result of the subsequence, the pulsed neurons in each case integrating values of sensor data of different input points in time of the subsequence;

feed the first processing result of the subsequence to a non-pulsed neural network, and

process the first processing result of the subsequence by non-pulsed neurons of one or multiple first layer(s) of the non-pulsed neural network for generating a second processing result of the subsequence;

feed the second processing results of the multiple subsequences to one or multiple second layer(s) of the non-pulsed neural network;

combine the second processing results of the multiple subsequences by non-pulsed neurons of the one or multiple second layer(s) of the non-pulsed neural network in that the non-pulsed neurons of the one or multiple second layer(s) in each case calculate a weighted sum of values of second processing results of different subsequences; and

ascertain the physical property of the physical object from outputs of the non-pulsed neurons of the one or multiple second layer(s), wherein a trigger determines when the processing by the non-pulsed neural network is executed, wherein the trigger corresponds to the pulsed neural network generating a number of pulses that meets a threshold amount of pulses.

8. A non-transitory computer-readable memory medium on which are stored program instructions for ascertaining a physical property of a physical object, the program instructions, when executed by one or multiple processors, causing the one or multiple processors to perform the following steps:

detecting sensor data using an event-based sensor for each input point in time of a sequence of input points in time, the sensor data including pieces of information about the physical object;

for each subsequence of a breakdown of the sequence of input points in time into multiple subsequences:

feeding the sensor data detected for the input points in time of the subsequence to a pulsed neural network,

processing the sensor data of the input points in time of the subsequence by pulsed neurons of the pulsed neural network for generating a first processing result of the subsequence, the pulsed neurons in each case integrating values of sensor data of different input points in time of the subsequence;

feeding the first processing result of the subsequence to a non-pulsed neural network, and

processing the first processing result of the subsequence by non-pulsed neurons of one or multiple first layer(s) of the non-pulsed neural network for generating a second processing result of the subsequence;

feeding the second processing results of the multiple subsequences to one or multiple second layer(s) of the non-pulsed neural network;

combining the second processing results of the multiple subsequences by non-pulsed neurons of the one or multiple second layer(s) of the non-pulsed neural network in that the non-pulsed neurons of the one or multiple second layer(s) in each case calculate a weighted sum of values of second processing results of different subsequences; and

ascertaining the physical property of the physical object from outputs of the non-pulsed neurons of the one or multiple second layer(s), wherein a trigger determines when the processing by the non-pulsed neural network is executed, wherein the trigger corresponds to the pulsed neural network generating a number of pulses that meets a threshold amount of pulses.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 14, 2022
From: KUGELE, ALEXANDER; PFEIFFER, MICHAEL; PFEIL, THOMAS
To: ROBERT BOSCH GMBH
Reel/Frame 058660/0497 →
Priority Claims (1)
DE 102020209538.8 · Jul 29, 2020 · national
Continuity (1)
Related Publication 20220036095A1 · Feb 3, 2022
References Cited (6)
US 9367798B2 · Coenen · 2016 [cited by examiner]
US 10339447B2 · Talathi · 2019 [cited by examiner]
US 20150206050A1 · Talathi · 2015 [cited by examiner]
US 20220126878A1 · Moustafa · 2022 [cited by examiner]
Lee, et al., “Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks”, (2020) arXiv, https://arxiv.org/abs/2003.06696v1, Mar. 2020 pp. 1-16 (Year: 2020). [cited by examiner]
Lee, et al.: “Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks”, (2003) arXiv, https://arxiv.org/abs/2003.06696v1, accessed Mar. 14, 2020, pp. 1-16. [cited by applicant]