System and method for high dynamic range depth capture using multiple cameras

1. A depth camera system comprising:

a processor;

a plurality of cameras coupled to the processor and configured to have overlapping fields of view, each camera being configured to detect light in a spectral band;

a projection source coupled to the processor and configured to emit light toward the fields of view of the cameras, wherein the projection source is configured to emit light having a pattern and in the spectral band; and

memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to:

control the cameras to capture, concurrently, a plurality of first images during a first exposure interval, each of the cameras concurrently capturing a corresponding one of the first images;

control the projection source to emit light at a first illumination level during the first exposure interval;

control the cameras to capture, concurrently, a plurality of second images during a second exposure interval, each of the cameras concurrently capturing a corresponding one of the second images;

control the projection source to emit light at a second illumination level during the second exposure interval, the second illumination level being different from the first illumination level;

compute at least one disparity map based on the first images and the second images;

generate a depth map from the at least one disparity map; and

output the depth map.

2. The depth camera system of claim 1 , wherein the first illumination level is a lower illumination level than the second illumination level.

3. The depth camera system of claim 2 , wherein the memory further stores instructions that, when executed by the processor, cause the processor to:

control the projection source to emit light at the first illumination level during the first exposure interval by emitting light for a first emission interval having a length shorter than the first exposure interval; and

control the projection source to emit light at the second illumination level during the second exposure interval by emitting light for a second emission interval having a length longer than the first emission interval and shorter than or equal to the second exposure interval.

4. The depth camera system of claim 2 , wherein the memory further stores instructions that, when executed by the processor, cause the processor to:

control the projection source to emit light at the first illumination level during the first exposure interval by emitting light at a first emission power; and

control the projection source to emit light at the second illumination level during the second exposure interval by emitting light at a second emission power greater than the first emission power.

5. The depth camera system of claim 1 , wherein the cameras are infrared cameras, and

wherein the projection source is configured to emit infrared light.

6. The depth camera system of claim 5 , wherein the projection source is configured to emit an infrared pattern.

7. The depth camera system of claim 1 , wherein the pattern is configured to be detectable at different illumination levels.

8. The depth camera system of claim 1 , wherein the cameras are configured to capture images using a global shutter.

9. The depth camera system of claim 1 , wherein the memory further stores instructions that, when executed by the processor, cause the processor to control the projection source to emit light at a third illumination level, the third illumination level being different from the first illumination level and different from the second illumination level.

10. The depth camera system of claim 1 , further comprising a second projection source, and

wherein the memory further stores instructions that, when executed by the processor, cause the processor to control the second projection source to emit light at a fourth illumination level different from the first illumination level and different from the second illumination level.

11. The depth camera system of claim 1 , wherein the memory further stores one or more previously captured images, and

wherein the memory further stores instructions that, when executed by the processor, cause the processor to:

analyze a brightness of at least one of the one or more previously captured images; and

set the second illumination level in accordance with a comparison of the brightness with a model brightness.

12. The depth camera system of claim 11 , wherein the projection source is configured to emit, selectively, one of a plurality of patterns at a time, each of the patterns corresponding to one of a plurality of illumination levels, and

wherein the second illumination level is set by selecting a pattern from the plurality of patterns.

13. The depth camera system of claim 1 , wherein the memory further stores one or more previously captured images, and

wherein the memory further stores instructions that, when executed by the processor, cause the processor to:

identify first portions of the first images having features matching second portions of the second images;

detect motion portions of the second images in accordance with locations of the first portions and the second portions; and

compute portions of the depth map corresponding to the motion portions in accordance with only the second images.

14. The depth camera system of claim 1 , wherein the memory further stores one or more previously captured images, and

wherein the memory further stores instructions that, when executed by the processor, cause the processor to compensate for motion using optical flow estimation by:

detecting portions of the images of the cameras subject to motion;

warping a previously captured image of the previously captured images; and

computing a depth estimate based on the second images and the warped previously captured images.

15. The depth camera system of claim 1 , wherein the projection source is configured to emit structured light.

16. A method for capturing a depth map in a scene having high dynamic range, comprising:

controlling, by a processor, a plurality of cameras to capture, concurrently, a plurality of first images during a first exposure interval, each camera being configured to detect light in a spectral band and each of the cameras concurrently capturing a corresponding one of the first images, the cameras having overlapping fields of view;

controlling, by the processor, a projection source to emit light at a first illumination level during the first exposure interval, wherein the projection source is configured to emit light having a pattern and in the spectral band;

controlling, by the processor, the cameras to capture, concurrently, a plurality of second images during a second exposure interval, each of the cameras concurrently capturing a corresponding one of the second images;

controlling, by the processor, the projection source to emit light at a second illumination level during the second exposure interval, the second illumination level being different from the first illumination level;

computing, by the processor, at least one disparity map based on the first images and the second images;

generate a depth map from the at least one disparity map; and

outputting, by the processor, the depth map.

17. The method of claim 16 , wherein the first illumination level is a lower illumination level than the second illumination level.

18. The method of claim 17 , further comprising:

controlling, by the processor, the projection source to emit light at the first illumination level during the first exposure interval by emitting light for a first emission interval having a length shorter than the first exposure interval; and

controlling, by the processor, the projection source to emit light at the second illumination level during the second exposure interval by emitting light for a second emission interval having a length longer than the first emission interval and shorter than or equal to the second exposure interval.

19. The method of claim 17 , further comprising:

controlling, by the processor, the projection source to emit light at the first illumination level during the first exposure interval by emitting light at a first emission power; and

controlling, by the processor, the projection source to emit light at the second illumination level during the second exposure interval by emitting light at a second emission power greater than the first emission power.

20. The method of claim 16 , wherein the cameras are infrared cameras, and

wherein the projection source is configured to emit infrared light.

21. The method of claim 20 , wherein the projection source is configured to emit an infrared pattern.

22. The method of claim 16 , wherein the pattern is configured to be detectable at different illumination levels.

23. The method of claim 16 , wherein the cameras are configured capture images using a global shutter.

24. The method of claim 16 , further comprising controlling, by the processor, the projection source to emit light at a third illumination level, the third illumination level being different from the first illumination level and different from the second illumination level.

25. The method of claim 16 , further comprising controlling a second projection source to emit light at a fourth illumination level different from the first illumination level and the second illumination level.

26. The method of claim 16 , further comprising:

storing, by the processor, the first images in a memory coupled to the processor;

analyzing, by the processor, a brightness of at least one of the first images; and

setting, by the processor, the second illumination second illumination level in accordance with a comparison of the brightness with a model brightness.

27. The method of claim 26 , wherein the projection source is configured to emit, selectively, one of a plurality of patterns at a time, each of the patterns corresponding to one of a plurality of illumination levels, and

wherein the second illumination level is set by selecting a pattern from the plurality of patterns.

28. The method of claim 16 , further comprising:

storing, by the processor, the first images and the second images in a memory coupled to the processor;

identifying, by the processor, first portions of the first images having features matching second portions of the second images;

detecting, by the processor, motion portions of the second images in accordance with locations of the first portions and the second portions; and

compute portions of the depth map corresponding to motion portions in accordance with only the second images.

29. The method of claim 16 , further comprising:

storing, by the processor, the first images and the second images in a memory coupled to the processor;

detecting, by the processor, portions of the images of the cameras subject to motion;

warping, by the processor, a previously captured image of a plurality of previously captured images; and

computing, by the processor, a depth estimate based on the second images and the warped previously captured images.

30. The method of claim 16 , wherein the projection source is configured to emit structured light.