IP Library › Granted Patent US 10,515,284
Granted Patent B2
US 10,515,284 · App. 15/413,390 · Granted Dec 24, 2019

Single-processor computer vision hardware control and application execution

Inventors: Evgeni Gousev (Saratoga, CA); Alok Govil (Fremont, CA); Jacek Maitan (Mountain View, CA); Nelson Rasquinha (San Diego, CA); Venkat Rangan (San Diego, CA); Edwin Chongwoo Park (San Diego, CA)
Assignee: QUALCOMM Incorporated
G06K9/00986G06K9/00255
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,515,284
App. No.
15/413,390
Granted
Dec 24, 2019
Kind
B2
Abstract

Apparatuses, methods, and systems are presented for reacting to scene-based occurrences. Such an apparatus may comprise dedicated computer vision (CV) computation hardware configured to receive sensor data from a sensor array comprising a plurality of sensor pixels and capable of computing one or more CV features using readings from neighboring sensor pixels of the sensor array. The apparatus may further comprise a first processing unit configured to control operation of the dedicated CV computation hardware. The first processing unit may be further configured to execute one or more application programs and, in conjunction with execution of the one or more application programs, communicate with at least one input/output (I/O) device controller, to effectuate an I/O operation in reaction to an event generated based on operations performed on the one or more computed CV features.

Claims (55)

1. An apparatus for reacting to scene-based occurrences, the apparatus comprising:

dedicated computer vision (CV) computation hardware configured to receive sensor data from a sensor array comprising a plurality of sensor pixels and capable of computing one or more CV features using readings from neighboring sensor pixels of the sensor array; and

a first processing unit configured to control operation of the dedicated CV computation hardware by specifying parameters associated with and affecting the computation of the one or more CV features,

wherein the first processing unit is configured to operate in a low-power state while the dedicated CV computation hardware computes the one or more CV features using readings from the neighboring sensor pixels of the sensor array, and

wherein the first processing unit is further configured to execute one or more application programs and, in conjunction with execution of the one or more application programs, communicate with at least one input/output (I/O) device controller, to effectuate an audio or visual I/O operation in reaction to a face detection event generated based on operations performed on the one or more computed CV features.

2. The apparatus of claim 1 , further comprising:

a classifier configured to receive an output of the dedicated CV computation hardware and perform the operations based on the one or more computed CV features, and

wherein the face detection event is generated in response to an output of the classifier.

3. The apparatus of claim 1 , wherein the dedicated CV computation hardware is configured to provide an indication of the face detection event to the first processing unit, in response to polling by the first processing unit.

4. The apparatus of claim 1 , wherein the face detection event is generated by the first processing unit.

5. The apparatus of claim 1 ,

wherein the first processing unit comprises a first core and a second core,

wherein the first core is configured to control the operation of the dedicated CV computation hardware; and

wherein the second core is configured to execute the one or more application programs.

6. The apparatus of claim 1 ,

wherein the first processing unit supports a software environment; and

wherein the software environment supports the one or more application programs.

7. The apparatus of claim 1 , wherein the I/O device controller comprises a media device controller.

8. The apparatus of claim 7 , wherein the media device controller is configured to generate an audio output.

9. The apparatus of claim 7 , wherein the media device controller is configured to generate a visual output.

10. The apparatus of claim 1 , wherein the I/O device controller comprises a proportional-integral-derivative (PID) controller.

11. The apparatus of claim 10 , wherein the PID controller is configured to generate a control signal to adjust a movable member to a desired position.

12. A method for reacting to scene-based occurrences, the method comprising:

receiving, at dedicated computer vision (CV) computation hardware, sensor data from a sensor array comprising a plurality of sensor pixels;

computing, at the dedicated CV computation hardware, one or more CV features using readings from neighboring sensor pixels of the sensor array;

controlling, using a first processing unit, operation of the dedicated CV computation hardware by specifying parameters associated with and affecting the computation of the one or more CV features,

wherein the first processing unit is operated in a low-power state while the one or more CV features are computed at the dedicated CV computation hardware using readings from the neighboring sensor pixels of the sensor array;

executing, at the first processing unit, one or more application programs; and

communicating, at the first processing unit and in conjunction with execution of the one or more application programs, with at least one input/output (I/O) device controller, to effectuate an audio or visual I/O operation in reaction to a face detection event generated based on operations performed on the one or more computed CV features.

13. The method of claim 12 , further comprising:

receiving, at a classifier, an output of the dedicated CV computation hardware and performing the operations based on the one or more computed CV features, and

wherein the face detection event is generated in response to an output of the classifier.

14. The method of claim 12 , wherein the dedicated CV computation hardware provides an indication of the face detection event to the first processing unit, in response to polling by the first processing unit.

15. The method of claim 12 , wherein the I/O device controller comprises a media device controller.

16. The method of claim 15 , wherein the media device controller is configured to generate an audio output.

17. The method of claim 15 , wherein the media device controller is configured to generate a visual output.

18. The method of claim 12 , wherein the I/O device controller comprises a proportional-integral-derivative (PID) controller.

19. The method of claim 18 , wherein the PID controller is configured to generate a control signal to adjust a movable member to a desired position.

20. An apparatus for reacting to scene-based occurrences, the apparatus comprising:

dedicated computer vision (CV) computation hardware configured to receive sensor data from a sensor array comprising a plurality of sensor pixels and capable of computing one or more CV features using readings from neighboring sensor pixels of the sensor array; and

a first processing unit configured to control the dedicated CV computation hardware by specifying parameters associated with and affecting the computation of the one or more CV features,

wherein the first processing unit is configured to operate in a low-power state while the dedicated CV computation hardware computes the one or more CV features using readings from the neighboring sensor pixels of the sensor array, and

wherein the first processing unit is further configured communicate with at least one device controller to control an audio or visual device in reaction to a face detection event generated based on the one or more computed CV features.

21. The apparatus of claim 20 , further comprising:

a classifier configured to receive an output of the dedicated CV computation hardware and perform operations based on the one or more computed CV features, and

wherein the face detection event is generated in response to an output of the classifier.

22. The apparatus of claim 20 , wherein the face detection event is generated by the first processing unit.

23. The apparatus of claim 20 , wherein the device controller comprises a proportional-integral-derivative (PID) controller.

24. The apparatus of claim 23 , wherein the PID controller is configured to generate a control signal to control the actuator device and adjust a movable member coupled to the actuator device to a desired position.

25. A method for reacting to scene-based occurrences, the method comprising:

receiving, at dedicated computer vision (CV) computation hardware, sensor data from a sensor array comprising a plurality of sensor pixels;

computing, at the dedicated CV computation hardware, one or more CV features using readings from neighboring sensor pixels;

controlling, using a first processing, the dedicated CV computation hardware by specifying parameters associated with and affecting the computation of the one or more CV features,

wherein the first processing unit is operated in a low-power state while the one or more CV features are computed at the dedicated CV computation hardware using readings from the neighboring sensor pixels of the sensor array; and

communicating, using the first processing unit, with at least one device controller, to control an audio or visual device in reaction to a face detection event generated based on the one or more computed CV features.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2017
From: GOUSEV, EVGENI; GOVIL, ALOK; MAITAN, JACEK; RASQUINHA, NELSON; RANGAN, VENKAT; PARK, EDWIN CHONGWOO
To: QUALCOMM INCORPORATED
Reel/Frame 043059/0118 →
Continuity (5)
Continuation In Part 14866549 · Sep 25, 2015
Provisional Application 62057800 · Sep 30, 2014
Provisional Application 62057972 · Sep 30, 2014
Provisional Application 62058009 · Sep 30, 2014
Related Publication 20170161579A1 · Jun 8, 2017
Cited By (1)
US 12,244,936