IP Library Granted Patent US 12,026,906
Granted Patent B1
US 12,026,906 · App. 18/096,219 · Granted Jul 2, 2024

Detector with synchronous event driven readout of pixels for direct time-of-flight depth sensing

Inventors: Augusto Ronchini Ximenes (Seattle, WA); Michael Hall (Bellevue, WA)
Assignee: Meta Platforms Technologies, LLC
G06T7/593G02B27/0172H04N13/128G02B2027/0178G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,026,906
App. No.
18/096,219
Granted
Jul 2, 2024
Kind
B1
Abstract

Each pixel of a plurality of pixels captures a portion of outgoing light illuminating a local area reflected from the one or more objects in the local area. A flag is generated via a flag determination logic circuit coupled to each pixel of the plurality of pixels. The flag indicates whether the reflected light was captured within a threshold amount of time from a current time. A time to digital converter is enabled via a flag determination logic circuit. The time to digital converter is associated with each pixel of the plurality of pixels to generate a digital representation of time when each pixel of the plurality of pixels captured the reflected light. Depth information is determined for the one or more objects in the local area based in part on the generated flag and the digital representation of time.

Claims (43)

1. A depth camera assembly (DCA) comprising:

an illumination source configured to illuminate a local area with outgoing light;

a detector including an array of macropixels, each macropixel in the array comprising a plurality of pixels, each pixel of the plurality of pixels configured to capture a portion of the outgoing light reflected from one or more objects in the local area, each pixel of the plurality of pixels coupled to a flag determination logic circuit that generates a flag indicating whether the reflected light was captured within a threshold amount of time from a current time, the flag determination logic circuit enabling a time to digital converter associated with each pixel of the plurality of pixels to generate a digital representation of time when each pixel of the plurality of pixels captured the reflected light; and

a controller configured to determine depth information for the one or more objects in the local area based in part on the generated flag and the digital representation of time.

2. The DCA of claim 1 , wherein each macropixel in the array includes a counter circuit coupled to the flag determination logic circuit, the counter circuit configured to maintain a count of a number of times the reflected light was captured within the threshold amount of time from the current time.

3. The DCA of claim 2 , wherein the controller is further configured to:

obtain the count from the counter circuit; and

provide a control signal to the detector identifying each macropixel in the array in response to the maintained count equaling or exceeding a threshold value.

4. The DCA of claim 1 , wherein each macropixel in the array is coupled to a compression circuit, the compression circuit configured to remove one or more digital timestamp signals from time to digital converters coupled to corresponding pixels of the plurality of pixels for which a corresponding flag determination circuit generated another flag indicating the corresponding pixels captured light from the local area greater than the threshold amount of time from the current time.

5. The DCA of claim 4 , wherein the compression circuit is further configured to output the other flag indicating the corresponding pixels captured the light greater than the threshold amount of time from the current time without the one or more digital timestamp signals.

6. The DCA of claim 4 , wherein the compression circuit is further configured to output the flag indicating a subset of the plurality of pixels captured light within the threshold amount of time from the current time in conjunction with at least one digital timestamp signal for the subset of pixels.

7. The DCA of claim 4 , wherein the compression circuit is coupled to a serialization circuit, the serialization circuit configured to combine data from multiple pixels of each macropixel in the array into a serial data stream transmitted to the controller via an output bus.

8. The DCA of claim 1 , wherein the detector further includes one or more access circuits configured to receive one or more control signals identifying a macropixel in the array.

9. The DCA of claim 8 , wherein the controller is further configured to:

provide the one or more control signals to the one or more access circuits identifying the macropixel;

receive data from the identified macropixel via an output bus; and

determine the depth information further based on the received data.

10. The DCA of claim 8 , wherein the one or more access circuits comprise:

a column access circuit coupled to a column of macropixels in the array; and

a plurality of row access circuits, each row access circuit coupled to a row of macropixels in the array.

11. The DCA of claim 10 , wherein the one or more control signals include a row access control signal identifying the row of macropixels and a column access control signal identifying the column of macropixels.

12. The DCA of claim 8 , wherein the one or more control signals include information identifying individual pixels within the identified macropixel.

13. The DCA of claim 1 , wherein the flag has a value indicating that the reflected light was captured within the threshold amount of time from the current time, and the flag has an another value indicating that the reflected light was captured greater than the threshold amount of time from the current time.

14. An imaging device comprising:

an array of macropixels, a respective macropixel in the array comprising a plurality of pixels, each pixel of the plurality of pixels configured to capture a portion of outgoing light reflected from one or more objects in a local area, each pixel of the plurality of pixels coupled to a flag determination logic circuit that generates a flag indicating whether the reflected light was captured within a threshold amount of time from a current time, the flag determination logic circuit enabling a time to digital converter associated with each pixel of the plurality of pixels to generate a digital representation of time when each pixel of the plurality of pixels captured the reflected light;

one or more access circuits configured to receive one or more control signals identifying the respective macropixel; and

an output bus configured to receive data from the time to digital converter, the data including the generated flag and the digital representation of time for determining depth information for the one or more objects in the local area.

15. The imaging device of claim 14 , wherein the respective macropixel includes a counter circuit coupled to the flag determination logic circuit, the counter circuit configured to maintain a count of a number of times the reflected light was captured within the threshold amount of time from the current time.

16. The imaging device of claim 14 , wherein the respective macropixel is coupled to a compression circuit, the compression circuit configured to remove one or more digital timestamp signals from time to digital converters coupled to corresponding pixels of the plurality of pixels for which a corresponding flag determination circuit generated another flag indicating the corresponding pixels captured light from the local area greater than the threshold amount of time from the current time.

17. The imaging device of claim 16 , wherein the compression circuit is coupled to a serialization circuit, the serialization circuit configured to combine data from multiple pixels of the respective macropixel into a serial data stream transmitted to a controller via the output bus for determining the depth information.

18. The imaging device of claim 14 , wherein the one or more access circuits comprise:

a column access circuit coupled to a column of macropixels in the array; and

a plurality of row access circuits, each row access circuit coupled to a row of macropixels in the array, and wherein

the one or more control signals include a row access control signal identifying the row of macropixels and a column access control signal identifying the column of macropixels, and

the one or more control signals include information identifying individual pixels within the respective macropixel.

19. A method comprising:

capturing, at each pixel of a plurality of pixels within each macropixel of an array of macropixels, a portion of outgoing light illuminating a local area reflected from one or more objects in the local area;

generating, via a flag determination logic circuit coupled to each pixel of the plurality of pixels, a flag indicating whether the reflected light was captured within a threshold amount of time from a current time;

enabling, via the flag determination logic circuit, a time to digital converter associated with each pixel of the plurality of pixels to generate a digital representation of time when each pixel of the plurality of pixels captured the reflected light; and

determining depth information for the one or more objects in the local area based in part on the generated flag and the digital representation of time.

20. The method of claim 19 , further comprising:

maintaining, via a counter circuit coupled to the flag determination logic circuit, a count of a number of times the reflected light was captured within the threshold amount of time from the current time; and

generating a control signal identifying each macropixel in the array in response to the maintained count equaling or exceeding a threshold value.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE CORRECT ASSIGNERS NAME FROM XIMENES, AUGUSTO RONCHINI TO RONCHINI XIMENS, AUGUSTO PREVIOUSLY RECORDED AT REEL: 67224 FRAME: 805. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded May 29, 2024
From: RONCHINI XIMENES, AUGUSTO; HALL, MICHAEL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 067557/0364 →
CORRECTIVE ASSIGNMENT TO CORRECT THE CORRECT THE CORRECTIVE ASSIGNMENT TO CORRECT ASSIGNORS NAME FROM XIMENES, AUGUSTO RONCHINI TO RONCHINI XIMENES, AUGUSTO PREVIOUSLY RECORDED AT REEL: 6224 FRAME: 805. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded May 29, 2024
From: RONCHINI XIMENES, AUGUSTO; HALL, MICHAEL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 067557/0529 →
CHANGE OF NAME Recorded May 13, 2024
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 067397/0648 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2024
From: XIMENES, AUGUSTO RONCHINI; HALL, MICHAEL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 067224/0805 →
Continuity (2)
Continuation 16838691 · Apr 2, 2020
Provisional Application 62828826 · Apr 3, 2019