Multimode detector for different time-of-flight based depth sensing modalities
A depth camera assembly (DCA) for multimode time-of-flight based depth sensing is presented herein. The DCA includes a projector, a detector, and a controller. The projector illuminates a target area with outgoing light comprising a plurality of light pulses. The detector includes an array of unit cells. Each unit cell includes a macropixel with a plurality of pixels that captures portions of the outgoing light reflected from the target area, and an array of memory cells coupled to the macropixel. At least one of the memory cells stores information about the captured portions of the reflected outgoing light received from the macropixel. The controller determines depth information for the target area based in part on data read from the at least one memory cell.
1. A depth camera assembly (DCA) comprising:
a projector configured to illuminate a target area with outgoing light comprising a plurality of light pulses;
a detector comprising an array of unit cells, each unit cell including:
a macropixel having a plurality of pixels configured to capture portions of the outgoing light reflected from the target area, and
an array of memory cells coupled to the macropixel, at least one memory cells configured to store information about the captured portions of the reflected outgoing light received from the macropixel; and
one or more access circuits that comprise:
a row decoder coupled to a charge based memory element of each memory cell providing a reference timing signal to the charge-based memory element; and
a column decoder coupled to the charge based memory element of each memory cell providing another reference timing signal to the charge-based memory element, wherein the row decoder and the column decoder are selectively activated or deactivated based at least in part on a desired resolution of the unit cell; and
a controller configured to provide one or more control signals to the detector identifying the at least one memory cell to determine depth information for the target area based in part on data read from the at least one memory cell.
2. The DCA of claim 1 , wherein the charge based memory element is associated with a capture bin having an adjustable time duration.
3. The DCA of claim 2 , wherein the controller is further configured to:
generate one or more capturing instructions based on information about a distance from the detector to the target area; and
adjust the time duration of the capture bin in accordance with the one or more capturing instructions.
4. The DCA of claim 1 , wherein the DCA is configured to operate in one of a direct time-of-flight (dTOF) mode, a photon-counting mode and an indirect time-of-flight (iTOF) mode for determination of the depth information, based in part on an activation state of the row decoder and an activation state of the column decoder.
5. The DCA of claim 4 , wherein the controller is further configured to:
activate the row decoder and the column decoder for configuring the DCA to operate in the dTOF mode;
initiate combining the captured portions of the reflected outgoing light stored in the plurality of pixels of the macropixel into a combined charge;
initiate sending the combined charge to a charge based memory element of a corresponding memory cell in the array of memory cells, based on the one or more control signals; and
determine the depth information for the target area based in part on the data read from the corresponding memory cell.
6. The DCA of claim 4 , wherein the controller is further configured to:
deactivate the row decoder or the column decoder based on the one or more control signals for configuring the DCA to operate in the iTOF mode;
initiate sending the captured portions of the reflected outgoing light from the plurality of pixels of the macropixel to at least one row of the memory cells or at least one column of the memory cells in accordance with reference timing signals provided to the at least one row or the at least one column via the row decoder or the column decoder; and
determine the depth information for the target area based in part on the data read from the at least one row or the at least one column of the memory cells.
7. The DCA of claim 4 , wherein the controller is further configured to:
deactivate the row decoder and the column decoder based on the one or more control signals for configuring the DCA to operate in the photon-counting mode;
initiate sending a captured portion of the reflected outgoing light from each pixel of the macropixel to a respective subset of the memory cells, based on the one or more control signals; and
determine the depth information for the target area based in part on the data read from the respective subset of the memory cells.
8. The DCA of claim 1 , wherein the detector further comprises an output bus configured to receive the data read from each charge based memory element in each of the at least one memory cell.
9. The DCA of claim 8 , wherein each charge based memory element comprises a charge sharing memory element configured to increment in fixed steps using a fixed charge transfer function, and the output bus is coupled to an analog-to-digital converter configured to correct non-linearity in the fixed charge transfer function.
10. The DCA of claim 1 , wherein the DCA is part of a headset and the plurality of light pulses from the projector illuminate the target area that includes a local area surrounding at least a portion of the headset.
11. A method comprising:
illuminating a target area with outgoing light comprising a plurality of light pulses;
capturing, at a macropixel in each unit cell of a detector, portions of the outgoing light reflected from the target area, the macropixel comprising a plurality of pixels;
storing information about the captured portions of the reflected outgoing light into at least one memory cell of an array of memory cells in each unit cell of the detector; and
determining depth information for the target area based in part on data read from the at least one memory cell, wherein the depth information is determined at least in in part by:
providing a reference timing signal to a charge based memory element of each memory cell via a row decoder coupled to the charge based memory element;
providing another reference timing signal to the charge based memory element via a column decoder coupled to the charge based memory element;
configuring the detector to operate in one of a direct time-of-flight (dTOF) mode, a photon-counting mode and an indirect time-of-flight (iTOF) mode for determination of the depth information, based in part on an activation state of the row decoder and an activation state of the column decoder.
12. The method of claim 11 , wherein each charge based memory element is associated with a capture bin having an adjustable time duration, and the method further comprising:
generating one or more capturing instructions based on information about a distance from the detector to the target area; and
adjusting the time duration of the capture bin in accordance with the one or more capturing instructions.
13. The method of claim 11 , further comprising:
activating the row decoder and the column decoder for configuring the dTOF mode;
combining the captured portions of the reflected outgoing light stored in the plurality of pixels of the macropixel into a combined charge;
sending the combined charge to a charge based memory element of a corresponding memory cell of the array of memory cells; and
determining the depth information for the target area based in part on the data read from the corresponding memory cell.
14. The method of claim 11 , further comprising:
deactivating the row decoder or the column decoder for configuring the iTOF mode;
sending the captured portions of the reflected outgoing light from the plurality of pixels of the macropixel to at least one row of the memory cells or at least one column of the memory cells in accordance with reference timing signals provided to the at least one row or the at least one column via the row decoder or the column decoder; and
determining the depth information for the target area based in part on the data read from the at least one row or the at least one column of the memory cells.
15. The method of claim 11 , further comprising:
deactivating the row decoder and the column decoder for configuring the photon-counting mode;
sending a captured portion of the reflected outgoing light from each pixel of the macropixel to a respective subset of the memory cells; and determining the depth information for the target area based in part on the data read from the respective subset of the memory cells.
16. The method of claim 11 , further comprising:
illuminating the target area with the light pulses emitted from an array of light emitters, each light emitter outputs a corresponding number of light pulses to a respective portion of the target area;
detecting, via an array of macropixels of the detector, portions of the light pulses reflected from the target area, each macropixel captures light over one or more respective timing windows from one or more different light emitters of the array of light emitters;
estimating signal-to-noise ratios (SNRs) for the array of macropixels;
updating a number of light pulses for at least one of the light emitters and a timing window for at least one of the macropixels, based on the estimated SNRs;
illuminating, via the array of light emitters, the target area with a second set of light pulses in accordance with the updated number of light pulses,
wherein the at least one light emitter outputs the updated number of light pulses;
detecting, via the array of macropixels, the second set of light pulses reflected from the target area, wherein the at least one macropixel captures light over a different timing window; and determining the depth information for the target area based in part on the detected second set of light pulses.
17. A detector comprising:
an array of unit cells, each unit cell including:
a macropixel having a plurality of pixels configured to capture portions of outgoing light reflected from a target area,
an array of memory cells configured to store information about the captured portions of the reflected outgoing light received from the macropixel, and
one or more access circuits configured to receive one or more control signals identifying at least one of the memory cells; and
an output bus configured to receive a data read from each charge based memory element in each of the at least one memory cell and wherein each charge based memory element comprises a charge sharing memory element configured to increment in fixed steps using a fixed charge transfer function, and the output bus is coupled to an analog-to-digital converter configured to correct non-linearity in the fixed charge transfer function; and
a controller configured to:
provide the one or more control signals to the one or more access circuits, and
determine depth information for the target area based in part on data read from the at least one identified memory cell.
18. The detector of claim 17 , wherein the controller is shared by all unit cells in the array.
19. The detector of claim 17 , wherein the controller is allocated to one unit cell in the array.