IP Library › Granted Patent US 10,147,024
Granted Patent B2
US 10,147,024 · App. 14/622,736 · Granted Dec 4, 2018

Interfacing an event based system with a frame based processing system

Inventors: Xin Wang (Shanghai, CN); William Howard Constable (San Diego, CA); Venkat Rangan (San Diego, CA); Manu Rastogi (San Diego, CA)
Assignee: QUALCOMM Incorporated
G06K9/6267G06K9/00624G06K9/20G06K9/605
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,147,024
App. No.
14/622,736
Granted
Dec 4, 2018
Kind
B2
Abstract

A method of interfacing an event based processing system with a frame based processing system is presented. The method includes converting multiple events into a frame. The events may be generated from an event sensor. The method also includes inputting the frame into the frame based processing system.

Claims (41)

1. A method of interfacing an event based processing system with a frame based processing system, comprising:

removing, at a filter that receives an event stream generated from an event sensor, an event from the event stream based on at least one of a spatial region of the event in a sensor array corresponding to pixels of the event sensor, a timing of the event in relation to a timing of another event of the event stream, or a combination thereof;

aggregating a plurality of events received from the event stream output from the filter;

converting the aggregated plurality of events into a frame that is compatible with the frame based processing system, which classifies a physical object represented by the aggregated plurality of events; and

inputting the frame into the frame based processing system for classification of the physical object based on features extracted from the frame, the features corresponding to the aggregated plurality of events.

2. The method of claim 1 , in which aggregating the plurality of events comprises aggregating the plurality of events for a specific time period.

3. The method of claim 2 , further comprising dynamically modifying the specific time period based at least in part on desired results from converting the plurality of events into the frame, processing load, or a combination thereof.

4. The method of claim 1 , in which aggregating the plurality of events comprises aggregating a specific number of events of the plurality of events.

5. The method of claim 4 , further comprising dynamically modifying the specific number of events based at least in part on desired results from converting the plurality of events into the frame, processing load, or a combination thereof.

6. The method of claim 1 , further comprising dynamically switching between a first interface for converting the plurality of events into the frame by aggregating events for a specific time period and a second interface for converting the plurality of events into the frame by aggregating a specific number of events.

7. The method of claim 1 , in which at least one event of the plurality of events is an indication from a vision sensor that a luminance has changed at a particular location.

8. An apparatus for interfacing an event based processing system with a frame based processing system, the apparatus comprising:

a memory unit; and

at least one processor coupled to the memory unit, the at least one processor being configured:

to remove, at a filter that receives an event stream generated from an event sensor, an event from the event stream based on at least one of a spatial region of the event in a sensor array corresponding to pixels of the event sensor, a timing of the event in relation to a timing of another event of the event stream, or a combination thereof;

to aggregate a plurality of events received from the event stream output from the filter;

to convert the aggregated plurality of events into a frame that is compatible with the frame based processing system, which classifies a physical object represented by the aggregated plurality of events; and

to input the frame into the frame based processing system for classification of the physical object based on features extracted from the frame, the features corresponding to the aggregated plurality of events.

9. The apparatus of claim 8 , in which the at least one processor is further configured to aggregate the plurality of events for a specific time period.

10. The apparatus of claim 9 , in which the specific time period is dynamically modified based at least on desired results from converting the plurality of events into the frame, processing load, or a combination thereof.

11. The apparatus of claim 8 , in which the at least one processor is further configured to aggregate a specific number of events of the plurality of events.

12. The apparatus of claim 11 , in which the specific number of events is dynamically modified based at least on desired results from converting the plurality of events into the frame, processing load, or a combination thereof.

13. The apparatus of claim 8 , in which the at least one processor is further configured to dynamically switch between a first interface for converting the plurality of events into the frame by aggregating events for a specific time period and a second interface for converting the plurality of events into the frame by aggregating a specific number of events.

14. The apparatus of claim 8 , in which at least one event of the plurality of events is an indication from a vision sensor that a luminance has changed at a particular location.

15. An apparatus for interfacing an event based processing system with a frame based processing system, the apparatus comprising:

means for filtering an event from an event stream, which is generated by an event sensor, based on at least one of a spatial region of the event in a sensor array corresponding to pixels of the event sensor, a timing of the event in relation to a timing of another event of the event stream, or a combination thereof;

means for aggregating a plurality of events received from the event stream output from the means for filtering;

means for converting the aggregated plurality of events into a frame that is compatible with the frame based processing system, which classifies a physical object represented by the aggregated plurality of events; and

means for inputting the frame into the frame based processing system for classification of the physical object based on features extracted from the frame, the features corresponding to the aggregated plurality of events.

16. The apparatus of claim 15 , in which the means for aggregating the plurality of events comprises means for aggregating the plurality of events for a specific time period.

17. The apparatus of claim 16 , in which the specific time period is dynamically modified based at least on desired results from converting the plurality of events into the frame, processing load, or a combination thereof.

18. The apparatus of claim 15 , in which the means for aggregating the plurality of events comprises means for aggregating a specific number of events of the plurality of events.

19. The apparatus of claim 18 , in which the specific number of events is dynamically modified based at least on desired results from converting the plurality of events into the frame, processing load, or a combination thereof.

20. The apparatus of claim 15 , further comprising means for dynamically switching between a first interface for converting the plurality of events into the frame by aggregating events for a specific time period and a second interface for converting the plurality of events into the frame by aggregating a specific number of events.

21. The apparatus of claim 15 , in which at least one event of the plurality of events is an indication from a vision sensor that a luminance has changed at a particular location.

22. A non-transitory computer-readable medium having program code recorded thereon for interfacing an event based processing system with a frame based processing system, the program code being executed by a processor and comprising:

program code to remove, at a filter that receives an event stream generated from an event sensor, an event from the event stream based on at least one of a spatial region of the event in a sensor array corresponding to pixels of the event sensor, a timing of the event in relation to a timing of another event of the event stream, or a combination thereof;

program code to aggregate a plurality of events received from the event stream output from the filter;

program code to convert the aggregated plurality of events into a frame that is compatible with the frame based processing system, which classifies a physical object represented by the aggregated plurality of events; and

program code to input the frame into the frame based processing system for classification of the physical object based on features extracted from the frame, the features corresponding to the aggregated plurality of events.

23. The method of claim 1 , in which removing the event based on the spatial region of the event comprises removing the event when a number of events at the spatial region of the event is greater than a threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2015
From: WANG, XIN; CONSTABLE, WILLIAM HOWARD; RANGAN, VENKAT; RASTOGI, MANU
To: QUALCOMM INCORPORATED
Reel/Frame 035871/0870 →
Continuity (2)
Provisional Application 62051145 · Sep 16, 2014
Related Publication 20160078321A1 · Mar 17, 2016