Depth measurement assembly with a structured light source and a time of flight camera

1. A depth measurement assembly (DMA) comprising:

an illumination source, wherein the illumination source is configured to project pulses of light at a temporal pulsing frequency into a local area, and a pulse of the pulses includes a structured light pattern;

a sensor configured to:

capture images of the pulses of light reflected from the local area, and

determine, using one or more of the captured images, one or more time of flight (TOF) phase shifts for the pulses of light; and

a controller coupled to the sensor and configured to:

determine a first set of estimated radial distances to an object in the local area based on the one or more TOF phase shifts,

determine a second estimated radial distance to the object via one or more triangulation calculations applied to one or more portions of at least one image of the captured images, and the at least one image is of the object illuminated by the pulse that includes the structured light pattern, and

select an estimated radial distance from the first set of radial distances, based in part on the second estimated radial distance.

2. The DMA of claim 1 , wherein the illumination source is configured to project pulses of structured light at a first temporal pulsing frequency at a first time, and at a second temporal pulsing frequency at a second time subsequent to the first time.

3. The DMA of claim 2 , wherein the sensor, for each of the first temporal pulsing frequency and the second temporal pulsing frequency, is configured to:

capture a first raw data using a first gated shutter window with a first timing shift relative to the pulsed illumination;

capture a second raw data in a second gated shutter window with a second timing shift relative to the pulsed illumination, and

capture a third raw data in a third gated shutter window with a first timing shift relative to the pulsed illumination.

4. The DMA of claim 3 , wherein the first raw data at the first temporal pulsing frequency is readout prior to capturing the second raw data, and the second raw data at the first temporal pulsing frequency is readout prior to capturing the third raw data at the first temporal pulsing frequency.

5. The DMA of claim 2 , wherein the sensor comprises a plurality of augmented pixels, and each augmented pixel includes three charge storage regions, wherein at the first temporal pulsing frequency, a first charge storage region collects charge associated with a first raw capture in a first gated shutter window with a first timing shift relative to the pulsed illumination, a second set of charge storage regions collects charge associated with the first raw capture in a second gated shutter window with a second timing shift relative to the pulsed illumination, and a third set of charge storage regions collects charge associated with the first raw capture in a third gated shutter window with a third timing shift relative to the pulsed illumination.

6. The DMA of claim 5 , wherein at the second temporal pulsing frequency the first charge storage region collects charge associated with a second raw capture in a first gated shutter window with a first timing shift relative to the pulsed illumination, the second set of charge storage regions collects charge associated with the second raw capture in a second gated shutter window with a second timing shift relative to the pulsed illumination, and the third set of charge storage regions collects charge associated with the second raw capture in a third gated shutter window with a third timing shift relative to the pulsed illumination.

7. The DMA of claim 6 , wherein the first raw capture at the first temporal pulsing frequency is readout prior to the second raw capture at the second temporal pulsing frequency.

8. The DMA of claim 1 , wherein the projected pulses of light composed of flood illumination overlaid with a structured light dot pattern, where each dot in the dot pattern has a brightness that is brighter than a brightness of the flood illumination.

9. The DMA of claim 1 , wherein the projected pulses of light composed of flood illumination overlaid with a structured light dot pattern, where each dot in the dot pattern has a brightness value that is less than a brightness value of the flood illumination.

10. The DMA of claim 1 , wherein a distance between the illumination source and the sensor is 50 mm or less.

11. The DMA of claim 1 , wherein the controller is further configured to obtain a depth map of the local area based on the selected estimated radial distance from the first set of radial distances.

12. A depth measurement assembly (DMA) comprising:

an illumination source, wherein the illumination source is configured to project pulses of light at a one or more temporal pulsing frequencies into a local area, and a pulse of the pulses includes a structured light pattern;

a sensor configured to:

capture images of the pulses of light reflected from the local area, and

determine, using one or more of the captured images, one or more time of flight (TOF) phase shifts for the pulses of light; and

an artificial reality device coupled to the sensor and configured to:

determine a first set of estimated radial distances to an object in the local area based on the one or more TOF phase shifts,

determine a second estimated radial distance to the object via one or more triangulation calculations applied to one or more portions of at least one image of the captured images, and the at least one image is of the object illuminated by the pulse that includes the structured light pattern, and

select an estimated radial distance from the first set of radial distances, based in part on the second estimated radial distance.

13. The DMA of claim 12 , wherein the artificial reality device is further configured to obtain a depth map of the local area based on the selected estimated radial distance from the first set of radial distances.

14. The DMA of claim 13 , wherein the artificial reality device is a head-mounted display (HMD) and the DMA is part of the HMD, the HMD comprising:

an electronic display element configured to display a virtual object based in part on the depth map of the local area, and

an optics block configured to direct light from the electronic display element to an exit pupil of the HMD.

15. The DMA of claim 12 , wherein the illumination source is configured to project pulses of structured light at a first temporal pulsing frequency at a first time, and at a second temporal pulsing frequency at a second time subsequent to the first time.

16. The DMA of claim 15 , wherein the sensor, for each of the first temporal pulsing frequency and the second temporal pulsing frequency, is configured to:

capture a first raw data using a first gated shutter window with a first timing shift relative to the pulsed illumination;

capture a second raw data in a second gated shutter window with a second timing shift relative to the pulsed illumination, and

capture a third raw data in a third gated shutter window with a first timing shift relative to the pulsed illumination.

17. The DMA of claim 16 , wherein the first raw data at the first temporal pulsing frequency is readout prior to capturing the second raw data, and the second raw data at the first temporal pulsing frequency is readout prior to capturing the third raw data at the first temporal pulsing frequency.

18. The DMA of claim 15 , wherein the sensor comprises a plurality of augmented pixels, and each augmented pixel includes 3 charge storage regions, wherein at the first temporal pulsing frequency, a first charge storage region collects charge associated with a first raw capture in a first gated shutter window with a first timing shift relative to the pulsed illumination, a second set of charge storage regions collects charge associated with the first raw capture in a second gated shutter window with a second timing shift relative to the pulsed illumination, and a third set of charge storage regions collects charge associated with the first raw capture in a third gated shutter window with a third timing shift relative to the pulsed illumination.

19. The DMA of claim 18 , wherein at the second temporal pulsing frequency the first charge storage region collects charge associated with a second raw capture in a first gated shutter window with a first timing shift relative to the pulsed illumination, the second set of charge storage regions collects charge associated with the second raw capture in a second gated shutter window with a second timing shift relative to the pulsed illumination, and the third set of charge storage regions collects charge associated with the second raw capture in a third gated shutter window with a third timing shift relative to the pulsed illumination.

20. The DMA of claim 19 , wherein the first raw capture at the first temporal pulsing frequency is readout prior to the second raw capture at the second temporal pulsing frequency.