IP Library Granted Patent US 10,883,821
Granted Patent B2
US 10,883,821 · App. 16/838,025 · Granted Jan 5, 2021

CMOS image sensor for 2D imaging and depth measurement with ambient light rejection

Inventor: Yibing Michelle Wang (Pasadena, CA)
G01B11/22G01B11/2518G03B15/02G06T7/521H04N5/2256H04N5/23245G03B17/54G06T2207/10028H04N13/271H04N2213/003
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,883,821
App. No.
16/838,025
Granted
Jan 5, 2021
Kind
B2
Abstract

Using the same image sensor to capture both a two-dimensional (2D) image of a three-dimensional (3D) object and 3D depth measurements for the object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor generates a multi-bit output in the 2D mode and a binary output in the 3D mode to generate timestamps. Strong ambient light is rejected by switching the image sensor to a 3D logarithmic mode from a 3D linear mode.

Claims (57)

1. A method, comprising:

capturing by a pixel of an image sensor an image of an object illuminated by at least a first light;

illuminating the object by a second light concurrently with the first light; and

determining a depth of the object using an output of the pixel,

the determining the depth of the object including:

generating a pixel-specific output from the pixel by performing one of the following:

generating the pixel-specific output by linearly integrating photoelectrons collected by the pixel over a predetermined time period based on an illumination level of the first light at a first level of illumination, and

generating the pixel-specific output based on a natural logarithm of an instantaneous photo-current produced by the pixel during the predetermined time period upon detecting the second light reflected from the object based on the illumination level of the first light at a second level of illumination that is greater than the first level of illumination; and

determining the depth of the object based on the pixel-specific output the pixel.

2. The method of claim 1 , wherein the first light comprises ambient light and the second light comprises laser light.

3. The method of claim 2 , wherein illuminating the object by the second light includes illuminating the object using a point scan with the second light.

4. The method of claim 1 , wherein a light source of the second light comprises one of a visible laser light source, a near infrared laser light source, a point light source, a monochromatic illumination source, and an X-Y addressable laser light source.

5. The method of claim 1 , wherein the image sensor comprises an array of pixels, and

wherein the pixel is a pixel of the array of pixels.

6. The method of claim 1 , wherein the first level of illumination is less than or equal to 10000 lux, and the second level of illumination is greater than 10000 lux.

7. The method of claim 1 , wherein capturing the image comprises:

generating a corresponding multi-bit output from an Analog-to-Digital Converter (ADC) associated with the pixel to capture the image; and

wherein determining the depth of the object comprises:

outputting from the ADC associated with the pixel a single bit output that triggers generation of a pixel-specific timestamp value by the ADC, the pixel-specific timestamp value being representative of a timing instance when the pixel detects the second light reflected from the object.

8. The method of claim 1 , wherein the pixel comprises one of a Four Transistor (4T) pixel, a Three Transistor (3T) pixel, a Two Transistor (2T) pixel, a One Transistor (1T) pixel, a shared-transistor pixel, a 1×2 2-shared pixel, and a 1×4 4-shared pixel.

9. The method of claim 1 , wherein the image sensor is a two-dimensional (2D) RGB (Red, Green, Blue) sensor, and wherein the pixel in the image sensor is operable in a 2D imaging mode and a three-dimensional (3D) imaging mode.

10. A method, comprising:

generating a pixel-specific output for a pixel of an image sensor corresponding to a first light reflected from an object that is concurrently illuminated by the first light and a second light; and

determining a depth of the object based at least in part on the pixel-specific output of the pixel by performing one of:

generating the pixel-specific output by linearly integrating photoelectrons collected by the pixel over a predetermined time period based on an illumination level of the second light at a first level of illumination, and

generating the pixel-specific output based on a natural logarithm of an instantaneous photo-current produced by the pixel during the predetermined time period upon detecting the first light reflected from the object based on the illumination level of the second light at a second level of illumination that is greater than the first level of illumination; and

determining the depth of the object based on the pixel-specific output the pixel.

11. The method of claim 10 , wherein the first light comprises laser light and the second light comprises ambient light.

12. The method of claim 10 , wherein the image sensor comprises an array of pixels, and

wherein the pixel is a pixel of the array of pixels.

13. The method of claim 10 , wherein the first level of illumination is less than or equal to 10000 lux, and the second level of illumination is greater than 10000 lux.

14. The method of claim 10 , wherein, for the pixel, generating the pixel-specific output by linearly integrating photoelectrons collected by the pixel includes:

turning on a reset signal to reset the pixel to a predetermined voltage level, and

turning off the reset signal during linear integration of photoelectrons collected by the pixel over the predetermined time period; and

wherein, for the pixel, generating the pixel-specific output based on a natural logarithm of an instantaneous photo-current includes:

turning on the reset signal to reset the pixel to the predetermined voltage level, and

maintaining the reset signal on during the predetermined time period to generate the pixel-specific output.

15. The method of claim 10 , wherein the image sensor comprises a two-dimensional (2D) RGB (Red, Green, Blue) sensor, and the pixel comprises one of a Four Transistor (4T) pixel, a Three Transistor (3T) pixel, a Two Transistor (2T) pixel, a One Transistor (1T) pixel, a shared-transistor pixel, a 1×2 2-shared pixel, and a 1×4 4-shared pixel.

16. A system, comprising:

an image sensor that captures an image of an object illuminated concurrently by a first light and a second light, wherein the image sensor comprises at least one pixel;

a processing unit that determines a depth of the object using the pixel, the processing unit determining the depth of the object based on an output of the pixel, the processing unit determined the depth of the object by:

determining an illuminance level of the first light;

receiving a pixel-specific output from the pixel of the image sensor as one of:

the pixel-specific output being linearly integrating photoelectrons collected by the pixel over a predetermined time period based on an illumination level of the first light at a first level of illumination, and

the pixel-specific output being based on a natural logarithm of an instantaneous photo-current produced by the pixel during the predetermined time period upon detecting the second light reflected from the object based on the illumination level of the first light at a second level of illumination that is greater than the first level of illumination; and

determining the depth of the object based on the pixel-specific output of the pixel.

17. The system of claim 16 , wherein the image sensor further comprises an array of pixels.

18. The system of claim 16 , wherein the first level of illumination is less than or equal to 10000 lux, and the second level of illumination is greater than 10000 lux.

19. The system of claim 16 , wherein the image sensor further comprises a plurality of Analog-to-Digital Converter (ADC) unit that is associated with the pixel, the ADC unit performing at least one of:

generating a multi-bit output for the pixel as part of capturing the image of the object; and

generating a single-bit output for the pixel when the depth of the object is to be determined.

20. The system of claim 19 , wherein to determine the depth of the object, the processing unit performs:

using the single-bit output of the ADC associated with the pixel to generate a pixel-specific timestamp value that is representative of a timing instance when the pixel detects the second light reflected from the object;

using the pixel-specific timestamp value to establish a correspondence between:

an offset distance between a collection optics associated with the image sensor and the pixel that detects the second light reflected light from the object, and

a scan angle used by a source of the second light to project the second light that is being detected by the pixel; and

determining the depth of the object based on the offset distance and the scan angle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2020
From: WANG, YIBING MICHELLE
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 054587/0043 →
Continuity (5)
Continuation 16186477 · Nov 9, 2018
Continuation 14842825 · Sep 1, 2015
Provisional Application 62182404 · Jun 19, 2015
Provisional Application 62150252 · Apr 20, 2015
Related Publication 20200232787A1 · Jul 23, 2020
Cited By (1)
US 12,376,752