Object recognition system and electronic apparatus
View Patent ↗An object recognition system according to an embodiment of the present disclosure includes an event detection sensor that detects, as an event, that a change in luminance of a pixel has exceeded a predetermined threshold, and a spiking neural network. The spiking neural network executes recognition processing on an object to be recognized on a basis of a result of the detection by the event detection sensor.
1 . An object recognition system, comprising:
an event detection sensor configured to detect, as an event, that a change in luminance of a pixel has exceeded a specific threshold;
an image sensor configured to acquire image information on an object to be recognized;
a ranging sensor configured to acquire three-dimensional image information on the object to be recognized; and
a spiking neural network, wherein
the spiking neural network includes
a first spiking neural network that executes a recognition process on the object to be recognized based on event information outputted from the event detection sensor,
a second spiking neural network that executes the recognition process based on the image information outputted from the image sensor, and
a third spiking neural network that executes the recognition process based on the three-dimensional image information outputted from the ranging sensor.
2 . The object recognition system according to claim 1 , wherein the event detection sensor includes an asynchronous imaging device.
3 . The object recognition system according to claim 1 , wherein the image sensor includes a synchronous imaging device configured to execute an imaging operation at a specific frame rate.
4 . The object recognition system according to claim 1 , wherein
the ranging sensor includes a Time-of-Flight (ToF) ranging device,
the ranging sensor is further configured to:
irradiate an object to be recognized with light, and
detect optical time-of-flight until return of the irradiated light reflected by the object to be recognized.
5 . The object recognition system according to claim 1 , further comprising a convolutional neural network that executes the recognition process based on at least one of the image information outputted from the image sensor or the three-dimensional image information outputted from the ranging sensor.
6 . The object recognition system according to claim 1 , wherein, in the event detection sensor, an event detection threshold of the event detection sensor is controlled based on a number of events at a periphery of a pixel that has outputted significant event information.
7 . The object recognition system according to claim 1 , wherein, in the event detection sensor,
a pixel group that has outputted significant event information is set as a region of interest, and
an event rate of a portion of the region of interest is increased.
8 . The object recognition system according to claim 1 , wherein, in the event detection sensor,
a pixel group that has outputted significant event information is set as a region of interest,
an event rate of a portion of the region of interest is increased, and
a frame rate of at least one of the image sensor or the ranging sensor is increased.
9 . The object recognition system according to claim 1 , further comprising a light source configured to irradiate an object to be recognized with light, wherein, in the event detection sensor,
a pixel group that has outputted significant event information is set as a region of interest,
an event rate of a portion of the region of interest is increased, and
the light source is controlled to irradiate the portion of the region of interest with light.
10 . An electronic apparatus, comprising;
an object recognition system,
the object recognition system including
an event detection sensor configured to detect, as an event, that a change in luminance of a pixel has exceeded specific threshold,
an image sensor configured to acquire image information on an object to be recognized,
a ranging sensor configured to acquire three-dimensional image information on the object to be recognized, and
a spiking neural network, wherein
the spiking neural network includes
a first spiking neural network that executes a recognition process on the object to be recognized based on event information outputted from the event detection sensor,
a second spiking neural network that executes the recognition process based on the image information outputted from the image sensor, and
a third spiking neural network that executes the recognition process based on the three-dimensional image information outputted from the ranging sensor.