Electronic device for compensating for time delay of dynamic vision sensor
Disclosed is an electronic device which includes a dynamic vision sensor that includes a first pixel sensing a change in light intensity and generates an event signal based on the sensed change in light intensity, an illuminance estimator that estimates illuminance of a light, and a time delay compensator that calculates a time delay between a first time at which the change in light intensity occurs and a second time at which the first pixel senses the change in light intensity, based on the illuminance of the light, and to compensate for the time delay.
1 . An image sensing device comprising:
a dynamic vision sensor including a first pixel configured to sense a change in light intensity, and configured to generate an event signal based on the change in the light intensity; and
a time delay compensator configured to calculate a time delay between a first time at which the change in the light intensity occurs and a second time at which the first pixel senses the change in the light intensity and to compensate for the time delay, the time delay compensator configured to calculate the time delay based on illuminance information, the illuminance information being based on a pixel illuminance for at least one second pixel.
2 . The image sensing device of claim 1 , wherein the event signal includes a location value of the first pixel, a polarity value including information about the change in the light intensity, and a timestamp value indicating a time at which the first pixel senses the change in the light intensity.
3 . The image sensing device of claim 2 , wherein the time delay compensator adjusts the timestamp value based on the time delay.
4 . The image sensing device of claim 1 , wherein the first pixel corresponds to the at least one second pixel.
5 . The image sensing device of claim 4 , wherein the at least one second pixel generates an electrical signal based on an incident light, and
wherein the pixel illuminance is based on the incident light.
6 . The image sensing device of claim 4 , wherein the at least one second pixel and the first pixel receive light reflected from a same object.
7 . The image sensing device of claim 1 , further comprising an image sensor configured to generate an electrical signal based on an incident light, and generate image data based on the electrical signal.
8 . The image sensing device of claim 7 , wherein the image sensor is implemented with a complementary metal-oxide-semiconductor (CMOS) image sensor.
9 . The image sensing device of claim 7 , wherein the image sensor includes the at least one second pixel,
wherein the illuminance information is based on the pixel illuminance for the at least one second pixel, and
wherein the first pixel corresponds to the at least one second pixel.
10 . The image sensing device of claim 9 , wherein the at least one second pixel generates the electrical signal,
wherein the image data include a pixel value corresponding to the electrical signal, and
wherein the pixel illuminance is calculated based on the pixel value.
11 . The image sensing device of claim 9 , wherein the at least one second pixel and the first pixel receive light reflected from a same object.
12 . The image sensing device of claim 1 , wherein the illuminance information includes illuminance of light that is received by the at least one second pixel, and
wherein, as the illuminance of the light decreases, the time delay increases, and the time delay compensator is configured to compensate for the increased time delay.
13 . The image sensing device of claim 1 , wherein the illuminance information includes a change in illuminance of light, the change in the illuminance of the light being calculated based on the illuminance of the light that is received by the at least one second pixel, and
wherein, as the change in the illuminance of the light decreases, the time delay increases, and the time delay compensator is configured to compensate for the increased time delay.
14 . An image sensing device comprising:
a dynamic vision sensor including a first pixel configured to sense a change in light intensity, and configured to generate a first event signal based on the change in the light intensity;
a complementary metal-oxide-semiconductor (CMOS) image sensor configured to generate image data based on an intensity of light; and
a time delay compensator configured to calculate a time delay between a first time at which the change in the light intensity occurs and a second time at which the first pixel senses the change in the light intensity, based on at least one of illuminance of the light or the change in light illuminance, and to generate a second event signal in which the time delay is compensated for,
wherein the illuminance of the light and the change in the light illuminance are calculated based on the image data, and
wherein the CMOS image sensor includes a second pixel.
15 . The image sensing device of claim 14 , wherein the dynamic vision sensor further includes a third pixel configured to sense the change in the light intensity, and generates a third event signal based on the change in the light intensity that is sensed by the third pixel,
wherein the CMOS image sensor further includes a fourth pixel,
wherein the first pixel corresponds to the second pixel, and
wherein the third pixel corresponds to the fourth pixel.
16 . The image sensing device of claim 15 , wherein the first event signal includes first time data,
wherein the third event signal includes second time data,
wherein the image data includes a first pixel value corresponding to a first electrical signal that the second pixel generates, and a second pixel value corresponding to a second electrical signal that the fourth pixel generates,
wherein the time delay compensator is configured to generate the third event signal that is obtained by compensating for a first time delay for the first time data based on at least one of a first pixel illuminance and a change in the first pixel illuminance, and generate a fourth event signal that is obtained by compensating for a second time delay for the second time data based on at least one of a second pixel illuminance or a change in the second pixel illuminance, and
wherein the at least one of the first pixel illuminance or the change in the first pixel illuminance are calculated based on the first pixel value, and the at least one of the second pixel illuminance or the change in the second pixel illuminance are calculated based on the second pixel value.
17 . A method of operating an image sensing device, comprising:
generating, by a dynamic vision sensor, event signal based on sensed change in light intensity;
calculating illuminance of a light or a change in the illuminance of the light based on the light that is reflected from an object and that is received by an image sensor including at least one pixel; and
compensating, by a time delay compensator, for a time delay between a first time at which the change in the light intensity occurs and a second time at which the dynamic vision sensor senses the change in the light intensity, based on the illuminance of the light or the change in the illuminance of the light.
18 . The method of operating the image sensing device of claim 17 , wherein the event signal includes a timestamp value indicating the second time, and further comprising,
adjusting the timestamp value to correspond to the first time.
19 . The method of operating the image sensing device of claim 17 , wherein the calculating the illuminance of the light or the change in the illuminance of the light based on the light that is reflected from the object and that is received by the image sensor including the at least one pixel includes:
generating, by the image sensor including the at least one pixel, image data based on the light reflected from the object;
estimating pixel illuminance based on a pixel value that is output by the at least one pixel; and
generating illuminance information comprising an information about the illuminance of the light or the change in the illuminance of the light, based on the pixel illuminance.