IP Library Granted Patent US 10,521,921
Granted Patent B2
US 10,521,921 · App. 15/476,100 · Granted Dec 31, 2019

Image capturing apparatus, system and method

Inventors: Xiaoyong Yang (San Jose, CA); Neale Dutton (Edinburgh, GB)
Assignees: STMICROELECTRONICS (RESEARCH & DEVELOPMENT) LIMITED; STMICROELECTRONICS, INC.
G06T7/55G01S7/48G01S7/4802G01S7/497G01S17/023G01S17/026G01S17/89G06K9/00255G06T7/521G06T7/74H04N5/2354G06T2207/10028G07C9/00158
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,521,921
App. No.
15/476,100
Granted
Dec 31, 2019
Kind
B2
Abstract

An apparatus includes time of flight single-photon avalanche diode (ToF SPAD) circuitry. The ToF SPAD circuitry generates indications of distance between the apparatus and an object within a field of view. A processor receives the indications of distance and controls at least one image sensor, such as a camera, to capture at least one image based on at least one indication of distance. The processor determines whether an image is a true representation of an expected object by comparing multiple indications of distance associated with the object to an expected object distance profile and comparing the image to at least one expected object image.

Claims (58)

1. An apparatus, comprising:

time of flight single-photon avalanche diode (ToF SPAD) circuitry, which, in operation, generates multiple indications of distance between the apparatus and at least one object within a field of view of the apparatus, wherein the multiple indications of distance corresponding to respective regions within the field of view of the apparatus; and

processing circuitry, which, in operation:

controls at least one image sensor to capture at least one image based on at least one of the multiple indications of distance between the apparatus and the at least one object within the field of view of the apparatus; and

determines whether the at least one image comprises a true representation of an expected object, the determining including:

comparing the multiple indications of distance to an expected object distance profile; and

comparing the at least one image to at least one expected object image, wherein the ToF SPAD circuitry, in operation, generates multiple indications of distance between the apparatus and the at least one object for a single region within the field of view, and the processing circuitry, in operation:

determines whether a translucent or transparent object is between the at least one image sensor and the at least one object in the field of view of the apparatus; and

controls an alert system based on the determining whether a translucent/transparent object is between the at least one image sensor and the at least one object in the field of view of the apparatus.

2. The apparatus as claimed in claim 1 wherein the processing circuitry, in operation:

determines whether the at least one object is within a field of view of the at least one image sensor based on at least one of the multiple indications of distance; and

controls the at least one image sensor to capture the at least one image based on the determination of whether the at least one object is within the field of view of the at least one image sensor, the at least one image comprising a representation of the object.

3. The apparatus as claimed in claim 2 wherein the processing circuitry, in operation, controls at least one illumination circuit based the determination of whether the at least one object is within the field of view of the at least one image sensor.

4. The apparatus as claimed in claim 1 wherein the processing circuitry, in operation:

controls at least one security function based on the comparison of the at least one image to the at least one expected object image.

5. The apparatus as claimed in claim 1 wherein the ToF SPAD circuitry, in operation, provides at least one of the multiple indications of distance to the at least one image sensor to set a focal distance of the at least one image sensor.

6. The apparatus as claimed in claim 1 wherein,

the at least one image sensor comprises at least two image sensors, the at least two image sensors having at least a partially overlapping field of view with a field of view of the ToF SPAD circuitry; and

the processing circuitry, in operation:

receives images from each of the at least two image sensors, the images comprising a representation of the object;

determines at least one indication of distance between the at least two image sensors and the at least one object within the field of view of the apparatus based on the images from each of the image sensors and parameters of the at least two image sensors; and

verifies the at least one indication of distance between the at least two image sensors and the at least one object based on at least one of the multiple indications of distance between the apparatus and the at least one object within the field of view of the apparatus.

7. The apparatus of claim 1 wherein the at least one image sensor comprises at least one camera.

8. A method, comprising:

generating, using time of flight single-photon avalanche diode (ToF SPAD) circuitry, multiple indications of distance between the ToF SPAD circuitry and at least one object within a field of view of the ToF SPAD circuitry, the multiple indications of distance corresponding to respective regions within the field of view of the ToF SPAD circuitry;

controlling, using processing circuitry, at least one image sensor to capture at least one image based on at least one of the multiple indications of distance between the ToF SPAD circuitry and the at least one object within the field of view of the ToF SPAD circuitry; and

determining, using the processing circuitry, whether the at least one image comprises a true representation of an expected object, the determining including:

comparing the multiple indications of distance to an expected object distance profile; and

comparing the at least one image to at least one expected object image, wherein the method comprises:

generating, using the ToF SPAD circuitry, multiple indications of distance between the ToF SPAD circuitry and the at least one object for a single region within the field of view:

determining, using the processing circuitry, whether a translucent or transparent object is between the at least one image sensor and the at least one object in the field of view; and

controlling an alert system based on the determining whether a translucent/transparent object is between the at least one image sensor and the at least one object in the field of view.

9. The method as claimed in claim 8 , comprising:

determining by the processing circuitry, whether the at least one object is within a field of view of the at least one image sensor based on at least one of the multiple indications of distance; and

controlling, by the processing circuitry, the at least one image sensor to capture the at least one image based on the determination of whether the at least one object is within the field of view of the at least one image sensor, the at least one image comprising a representation of the object.

10. The method as claimed in claim 9 , comprising:

controlling, by the processing circuitry, at least one illumination circuit based the determination of whether the at least one object is within the field of view of the at least one image sensor.

11. The method as claimed in claim 8 , comprising controlling at least one security function based on the comparison of the at least one image to the at least one expected object image.

12. The method of claim 8 , comprising providing at least one of the multiple indications of distance to the at least one image sensor to set a focal distance of the at least one image sensor.

13. The method of claim 8 wherein the at least one image sensor comprises at least two image sensors, the at least two image sensors having at least a partially overlapping field of view with a field of view of the ToF SPAD circuitry, and the method comprises:

controlling, using the processing circuitry, the at least two image sensors to capture respective images comprising a representation of the at least one object;

determining, using the processor, at least one indication of distance between the at least two image sensors and the at least one object within the field of view based on the images from each of the at least two image sensors and parameters of the at least two image sensors; and

verifying the at least one indication of distance between the at least two image sensors and the at least one object within the field of view based on at least one of the multiple indications of distance between the apparatus and the at least one object in the field of view of the apparatus.

14. The method of claim 8 wherein the at least one image sensor comprises at least one camera.

15. A system, comprising:

image sensing circuitry, which, in operation, captures images of objects;

time of flight single-photon avalanche diode (ToF SPAD) circuitry, which, in operation, generates indications of distance associated with objects within a field of view, the indications of distance corresponding to respective regions within the field of view; and

processing circuitry, which, in operation:

controls capturing of images by the image sensing circuitry based on indications of distance associated with objects generated by the ToF SPAD circuitry; and

determines whether an image comprises a true representation of an expected object, the determining including:

comparing multiple indications of distance associated with the object to an expected object distance profile; and

comparing the image to at least one expected object image, wherein the ToF SPAD circuitry, in operation, generates multiple indications of distance associated with at least one object for a single region within the field of view, and the processing circuitry, in operation:

determines whether a translucent or transparent object is between the image sensing circuitry and the at least one object in the field of view; and

controls an alert system based on the determining whether a translucent/transparent object is between the image sensing circuitry and the at least one object in the field of view.

16. The system of claim 15 , comprising illumination circuitry, wherein the processing circuitry, in operation, controls the illumination circuitry based on indications of distance associated with objects generated by the ToF SPAD circuitry.

17. The system of claim 15 , comprising an integrated circuit, the integrated circuit including at least the ToF SPAD circuitry and the image sensing circuitry.

18. The system of claim 17 wherein the image sensing circuitry comprises at least two image sensors, the at least two image sensors having at least a partially overlapping field of view with a field of view of the ToF SPAD circuitry.

19. The system of claim 15 , comprising smart phone circuitry.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 11, 2017
From: YANG, XIAOYONG
To: STMICROELECTRONICS, INC.
Reel/Frame 042340/0004 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 11, 2017
From: DUTTON, NEALE
To: STMICROELECTRONICS (RESEARCH & DEVELOPMENT) LIMITED
Reel/Frame 042340/0045 →
Priority Claims (1)
EP 16191039 · Sep 28, 2016 · regional
Continuity (1)
Related Publication 20180089848A1 · Mar 29, 2018