IP Library Granted Patent US 12,034,015
Granted Patent B2
US 12,034,015 · App. 16/421,441 · Granted Jul 9, 2024

Programmable pixel array

Inventors: Andrew Samuel Berkovich (Bellevue, WA); Xinqiao Liu (Medina, WA); Richard Andrew Newcombe (Seattle, WA); Song Chen (Redmond, WA); Wei Gao (Bothell, WA)
Assignee: Meta Platforms Technologies, LLC
H01L27/14605H04N5/23229H04N5/341
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,034,015
App. No.
16/421,441
Granted
Jul 9, 2024
Kind
B2
Abstract

Methods and systems for performing light measurement are disclosed. In one example, an apparatus comprises an array of pixel cells, each pixel cell of the array of pixel cells configured to perform a light measurement operation and to generate a digital output of the light measurement operation. The apparatus further includes a peripheral circuit configured to: receive a pixel array programming map including programming data targeted at each pixel cell of the array of pixel cells, and configure the light measurement operation at the each pixel cell based on the programming data targeted at the each pixel cell. The apparatus further includes an image processor configured to generate an image frame based on the digital outputs of at least some of the array of pixel cells.

Claims (78)

1. An apparatus comprising:

an array of pixel cells, each pixel cell of the array of pixel cells configured to perform a light measurement operation and to generate a digital output of the light measurement operation;

a peripheral circuit configured to:

receive a pixel array programming map including programming data targeted at each pixel cell of the array of pixel cells; and

configure the light measurement operation at the each pixel cell based on the programming data targeted at the each pixel cell; and

an image processor configured to generate an image frame based on the digital outputs of at least some of the array of pixel cells,

wherein the peripheral circuit is further configured to disable generation of digital pixel data from one or more pixel cells of the array of pixel cells based on the pixel array programming map.

2. The apparatus of claim 1 , wherein the each pixel cell of the array of pixel cells is associated with a configuration memory that is individually addressable;

wherein the peripheral circuit is configured to:

extract first programming data for a first pixel cell of the array of pixel cells from the pixel array programming map;

extract second programming data for a second pixel cell of the array of pixel cells from the pixel array programming map;

generate a first address based on the first programming data;

generate a second address based on the second programming data;

select, based on the first address, the configuration memory of the first pixel cell to receive the first programming data; and

select, based on the second address, the configuration memory of the second pixel cell to receive the second programming data; and

wherein the first programming data and the second programming data are different.

3. The apparatus of claim 2 , wherein the pixel array programming map comprises an array of programming data; and

wherein the first address is generated based on a location of the first programming data within the array of programming data.

4. The apparatus of claim 2 , wherein the each pixel cell includes at least one device to control the generation of the digital output; and

wherein the peripheral circuit is configured to:

transmit, based on the first address, a first signal to the at least one device of the first pixel cell to enable the generation of a first digital output at the first pixel cell; and

transmit, based on the second address, a second signal to the at least one device of the second pixel cell to disable the generation of a second digital output at the second pixel cell.

5. The apparatus of claim 4 , wherein the peripheral circuit is configured to disable one or more sense amplifiers configured to amplify the second digital output based on the second address.

6. The apparatus of claim 4 , wherein the peripheral circuit is configured to output a stream of digital outputs including the first digital output to the image processor; and

wherein the image processor is configured to:

synchronize timing of reception of the stream of digital outputs based on the pixel array programming map; and

identify the first digital output from the stream of digital outputs based on the synchronization.

7. The apparatus of claim 4 , wherein the first programming data and the second programming data are generated based on a region of interest determined from a prior image frame.

8. The apparatus of claim 2 , wherein the each pixel cell is configured to perform the light measurement operation within a programmable exposure period;

wherein the peripheral circuit is configured to:

transmit, based on the first address, a first signal to the first pixel cell to set a first exposure period for a first light measurement operation; and

transmit, based on the second address, a second signal to the second pixel cell to set a second exposure period for a second light measurement operation, the second exposure period being different from the first exposure period; and

wherein the image processor is configured to receive a first digital output of the first light measurement operation and a second digital output of the second light measurement operation.

9. The apparatus of claim 8 , wherein the peripheral circuit is configured to transmit, based on the first address, the first signal to the first pixel cell to perform the first light measurement operation based on at least one of: setting the first exposure period to zero or putting the first pixel cell in a reset state;

wherein the image processor is configured to:

perform a calibration operation based on the first digital output; and

determine an intensity of light received by the second pixel cell based on the second digital output and a result of the calibration operation.

10. The apparatus of claim 2 , wherein the each pixel cell is configured to generate an analog signal representing a result of the light measurement operation, the digital output being generated based on quantizing the analog signal at a programmable quantization resolution; and

wherein the peripheral circuit is configured to:

transmit, based on the first address, a first signal to the first pixel cell to set a first quantization resolution; and

transmit, based on the second address, a second signal to the second pixel cell to set a second quantization resolution different from the first quantization resolution.

11. The apparatus of claim 10 , wherein the each pixel cell includes a counter configured to generate the digital output based on comparing the analog signal with a ramping signal, the quantization resolution being programmable based on a frequency of a clock signal supplied to the counter; and

wherein the peripheral circuit is configured to:

transmit, based on the first address, a first clock signal of a first clock frequency to the first pixel cell to set the first quantization resolution; and

transmit, based on the second address, a second clock signal of a second clock frequency to the second pixel cell to set the second quantization resolution.

12. The apparatus of claim 2 , wherein the first pixel cell includes a summation circuit configured to generate a summation signal based on summing a first analog signal from a first light measurement operation at the first pixel cell and a second analog signal from a second light measurement operation at the second pixel cell;

wherein the peripheral circuit is configured to:

transmit, based on the first address, a first signal to configure the first pixel cell to generate a first digital output based on quantizing the summation signal; and

transmit, based on the second address, a second signal to the second pixel cell to disable generation of a second digital output by the second pixel cell.

13. The apparatus of claim 2 , wherein the each pixel includes a first photodiode configured to measure light of a first wavelength range and a second photodiode to measure light of a second wave length range; and

wherein the peripheral circuit is configured to:

transmit, based on the first address, a first signal to the first pixel cell to generate a first digital output based on output from the first photodiode; and

transmit, based on the second address, a second signal to the second pixel cell to generate a second digital output based on an output from the second photodiode.

14. The apparatus of claim 1 , further comprising a post processor circuit configured to perform post processing of at least some of the digital outputs of the array of pixel cells,

wherein the image processor is configured to generate the image frame based on the post processed at least some of the digital outputs of the array of pixel cells; and

wherein the post processor circuit, the peripheral circuit, and the array of pixel cells are integrated on a semiconductor substrate.

15. The apparatus of claim 14 , wherein the array of pixel cells is formed on a first semiconductor substrate;

wherein the peripheral circuit and the post processor circuit are formed on a second semiconductor substrate; and

wherein the first semiconductor substrate and the second semiconductor substrate forms a stack.

16. The apparatus of claim 14 , wherein the post processor circuit is configured to:

extract, from the at least some of the digital outputs of the array of pixel cells, features corresponding to reflection of structured light by an object;

based on a result of the extraction, identify pixel cells of which the digital outputs represent the extracted features; and

transmit only the digital outputs of the identified pixel cells to the image processor.

17. The apparatus of claim 16 , wherein the post processor circuit is configured to:

determine sub pixel locations of the extracted features; and

provide the sub pixel locations to an application to perform a depth sensing operation of the object.

18. The apparatus of claim 15 , wherein the post processor circuit comprises the image processor and is configured to:

receive the digital outputs generated by the array of pixel cells at a first frame rate; and

generate image frames based on the digital outputs at a second frame rate lower than the first frame rate.

19. The apparatus of claim 1 , further comprising an illuminator configured to project light to different parts of a scene based on an illumination sequence; and

wherein the pixel array programming map is generated based on the illumination sequence.

20. A method comprising:

receiving a pixel array programming map including programming data targeted at each pixel cell of an array of pixel cells;

extracting first programming data for a first pixel cell of the array of pixel cells from the pixel array programming map;

extracting second programming data for a second pixel cell of the array of pixel cells from the pixel array programming map;

configuring a first light measurement operation at the first pixel cell based on the first programming data;

configuring a second light measurement operation at the second pixel cell based on the second programming data; and

generating an image frame based on digital pixel data output by at least one of the first pixel cell or the second pixel cell, wherein at least one of the first or second light measurement operations comprises disabling generation of digital pixel data from the respective first or second pixel cell.

Assignments (2)
CHANGE OF NAME Recorded Jul 27, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060990/0518 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2019
From: BERKOVICH, ANDREW SAMUEL; LIU, XINQIAO; NEWCOMBE, RICHARD ANDREW; CHEN, SONG; GAO, WEI
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 050087/0030 →
Continuity (2)
Provisional Application 62676849 · May 25, 2018
Related Publication 20190363118A1 · Nov 28, 2019
Cited By (1)
US 12,282,170