IP Library Granted Patent US 10,531,031
Granted Patent B2
US 10,531,031 · App. 15/914,073 · Granted Jan 7, 2020

Image processing device configured to regenerate timestamp and electronic device including the same

Inventors: Keun Joo Park (Seoul, KR); Junseok Kim (Hwaseong-si, KR); Yohan Roh (Suwon-si, KR); Hyunsurk Ryu (Hwaseong-si, KR); Jooyeon Woo (Hwaseong-si, KR); Hyunku Lee (Suwon-si, KR)
Assignee: Samsung Electronics Co., Ltd.
H04N5/3745G06F16/2322G06F16/583H04N5/341H04N5/3675H04N5/37455H05B39/042H05B37/029H05B37/0218H05B37/0281Y02B20/14
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,531,031
App. No.
15/914,073
Granted
Jan 7, 2020
Kind
B2
Abstract

An image processing device includes a vision sensor and a processor. The vision sensor generates a plurality of events in which an intensity of light changes and generates a plurality of timestamps depending on times when the events occur. In addition, the processor may regenerate a timestamp of a pixel where an abnormal event occurs, based on temporal correlation of the events.

Claims (57)

1. An image processing device comprising:

a vision sensor configured to generate a plurality of events through at least one pixel of a plurality of pixels, and generate a plurality of timestamps associated with times when the plurality of events occur, each of the events corresponding to a change in intensity of light observed by the at least one pixel of the plurality of pixels, the plurality of pixels including a target pixel and adjacent pixels around the target pixel; and

at least one processor configured to,

classify the plurality of events into at least a first group and a second group based on values of the timestamps associated with the events, the events associated with the first group associated with a first timestamp value, and the events associated with the second group associated with a second timestamp value,

determine a direction in which the plurality of events occur,

determine whether at least one event associated with the target pixel or the adjacent pixels is an abnormal event based on the determined direction and the timestamp values corresponding to the target pixel and the adjacent pixels, the determining whether the at least one event is the abnormal event including determining whether the at least one event is associated with a third group, the third group having a third timestamp value, and

replace the third timestamp value with any one of timestamps values corresponding to pixels adjacent to a pixel causing the abnormal event.

2. The image processing device of claim 1 , wherein the at least one processor is further configured to:

replace the third timestamp value with a smallest value of the timestamps corresponding to the pixels adjacent to the pixel causing the abnormal event.

3. The image processing device of claim 1 , wherein the at least one processor is further configured to:

replace the third timestamp value with a “0” value.

4. The image processing device of claim 1 , wherein the at least one processor is further configured to:

determine an outline of an object based on the first group and the second group, the object causing generation of the plurality of events.

5. The image processing device of claim 1 , wherein a first event with the first timestamp value occurs prior to at least one second event with the second timestamp value.

6. The image processing device of claim 1 , further comprising:

a first pixel configured to generate a first event; and

at least one second pixel configured to generate at least one second event, the at least one second pixel adjacent to the first pixel.

7. The image processing device of claim 6 , wherein the pixel causing the abnormal event is not adjacent to the second pixel.

8. The image processing device of claim 1 , wherein the vision sensor includes:

a pixel array including the plurality of pixels arranged in rows and columns; and

an event detection circuit configured to,

access the plurality of pixels corresponding to the plurality of events from among the plurality of pixels of the pixel array, and

generate the timestamps associated with the plurality of events, the timestamps including the times when the events occur.

9. An image processing device comprising:

a vision sensor configured to generate a plurality of events through at least one pixel of a plurality of pixels, and generate a plurality of timestamps associated with times when the plurality of events occur, each of the events corresponding to a change in intensity of light observed by the at least one pixel of the plurality of pixels, the plurality of pixels including a target pixel and adjacent pixels around the target pixel, the plurality of timestamps including at least a first timestamp corresponding to at least one first event that occurs in a first time period, and a second timestamp corresponding to at least one second event that occurs in a second time period; and

at least one processor configured to,

determine whether the target pixel is a dead pixel based on a temporal correlation between at least one pixel associated with the first timestamp and at least one pixel associated with the second timestamp, and

replace a timestamp of the target pixel with any one of timestamps of events that occurred at the adjacent pixels based on results of the determination.

10. The image processing device of claim 9 , wherein the at least one processor is further configured to:

replace the timestamp of the target pixel with a timestamp having a greatest value among the timestamps of the events that occurred at the adjacent pixels.

11. The image processing device of claim 9 , wherein the at least one processor is further configured to:

replace the timestamp of the target pixel with a timestamp of the at least one second event.

12. The image processing device of claim 9 , wherein the at least one processor is further configured to:

determine an outline of an object based on the at least one first event and the at least one second event, the object causing generation of the plurality of events.

13. The image processing device of claim 9 , wherein the at least one first event occurs prior to the at least one second event.

14. The image processing device of claim 9 , further comprising:

at least one first pixel configured to generate the at least one first event; and

at least one second pixel configured to generate the at least one second event, the at least one second pixel adjacent to the at least one first pixel.

15. An image processing device comprising:

a vision sensor configured to generate a plurality of events through at least one pixel of a plurality of pixels, and generate a plurality of timestamps associated with times when the plurality of events occur, each of the events corresponding to a change in intensity of light observed by the at least one pixel, the plurality of pixels classified into a plurality of pixel groups, each pixel group including at least two pixels adjacent to each other; and

at least one processor configured to,

select a first representative value from timestamps corresponding to a first pixel group of the plurality of pixel groups,

select a second representative value from timestamps corresponding to a second pixel group of the plurality of pixel groups,

select a third representative value from timestamps corresponding to a third pixel group of the plurality of pixel groups,

determine whether the third representative value of the third pixel group comes from an abnormal event based on a temporal correlation between the first representative value and the second representative value, and

replace the third representative value with any representative value corresponding to pixel groups adjacent to the third pixel group.

16. The image processing device of claim 15 , wherein the at least one processor is further configured to:

replace the third representative value with a representative value having a greatest value among the representative values corresponding to the pixel groups adjacent to the third pixel group.

17. The image processing device of claim 15 , wherein the at least one processor is further configured to:

replace the third representative value with a representative value having a smallest value among the representative values corresponding to the pixel groups adjacent to the third pixel group.

18. The image processing device of claim 15 , wherein the at least one processor is further configured to:

replace the third representative value with a “0” value.

19. The image processing device of claim 15 , wherein the at least one processor is further configured to:

determine an outline of an object based on the first representative value, the second representative value, and the replaced third representative value, the object causing generation of the plurality of events.

20. The image processing device of claim 15 , wherein

the first pixel group and the second pixel group are adjacent to each other; and

the third pixel group is not adjacent to the first pixel group and the second pixel group.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2018
From: PARK, KEUN JOO; KIM, JUNSEOK; ROH, YOHAN; RYU, HYUNSURK; WOO, JOOYEON; LEE, HYUNKU
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 045134/0531 →
Priority Claims (2)
KR 10-2017-0029687 · Mar 8, 2017 · national
KR 10-2017-0145538 · Nov 2, 2017 · national
Continuity (1)
Related Publication 20180262705A1 · Sep 13, 2018
Cited By (1)
US 12,236,653