Time-of-flight (TOF) assisted structured light imaging

1. A method for computing a depth map of a scene in a structured light imaging system comprising a time-of-flight (TOF) sensor and a projector configured to project a plurality of phase-shifted structured light patterns, the method comprising:

capturing a plurality of high frequency phase-shifted structured light images of the scene using a camera in the structured light imaging system;

generating, concurrently with the capturing of the plurality of high frequency phase-shifted structured light images, a time-of-flight (TOF) depth image of the scene using the TOF sensor; and

computing the depth map from the plurality of high frequency phase-shifted structured light images by:

mapping a location in a plane of the projector to a first location in an image of the camera using the TOF depth image, wherein the TOF depth image is used for phase unwrapping;

searching a neighborhood of locations in the plurality of high frequency phase-shifted structured light images around the first location to find a second location corresponding to the projector plane location, wherein the second location corresponds to the projector plane location when a phase value of the second location is equal to a phase value of the projector plane location; and

computing a depth value for the projector location based on the second location.

2. The method of claim 1 , wherein mapping a location comprises:

mapping the projector plane location to a location in the TOF depth image using a projector-TOF sensor calibration matrix; and

mapping the location in the TOF depth image to the first location using a camera-TOF sensor calibration matrix.

3. The method of claim 1 , wherein the camera is the TOF sensor.

4. The method of claim 3 , wherein the projector is a digital micromirror device using a phase modulated light emitting diode as a light source.

5. A structured light imaging system comprising:

a projector configured to project a plurality of phase-shifted structured light patterns; and

a time-of-flight (TOF) sensor,

wherein the structured light imaging system is configured to compute a depth map of a scene by performing a method comprising:

capturing a plurality of high frequency phase-shifted structured light images of the scene using a camera in the structured light imaging system;

generating, concurrently with the capturing of the plurality of high frequency phase-shifted structured light images, a time-of-flight (TOF) depth image of the scene using the TOF sensor; and

computing the depth map from the plurality of high frequency phase-shifted structured light images by:

mapping a location in a plane of the projector to a first location in an image of the camera using the TOF depth image, wherein the TOF depth image is used for phase unwrapping;

searching a neighborhood of locations in the plurality of high frequency phase-shifted structured light images around the first location to find a second location corresponding to the projector plane location, wherein the second location corresponds to the projector plane location when a phase value of the second location is equal to a phase value of the projector plane location; and

computing a depth value for the projector location based on the second location.

6. The structured light imaging system of claim 5 , wherein mapping a location comprises:

mapping the projector plane location to a location in the TOF depth image using a projector-TOF sensor calibration matrix; and

mapping the location in the TOF depth image to the first location using a camera-TOF sensor calibration matrix.

7. The structured light imaging system of claim 5 , wherein the camera is the TOF sensor.

8. The structured light imaging system of claim 7 , wherein the projector is a digital micromirror device using a phase modulated light emitting diode as a light source.

9. A non-transitory computer readable medium storing software instructions that, when executed by at least one processor in a structured light imaging system comprising a time-of-flight (TOF) sensor and a projector configured to project a plurality of phase-shifted structured light patterns, cause execution of a method for computing a depth map of a scene, the method comprising:

capturing a plurality of high frequency phase-shifted structured light images of the scene using a camera in the structured light imaging system;

generating, concurrently with the capturing of the plurality of high frequency phase-shifted structured light images, a time-of-flight (TOF) depth image of the scene using the TOF sensor; and

computing the depth map from the plurality of high frequency phase-shifted structured light images by:

mapping a location in a plane of the projector to a first location in an image of the camera using the TOF depth image, wherein the TOF depth image is used for phase unwrapping;

searching a neighborhood of locations in the plurality of high frequency phase-shifted structured light images around the first location to find a second location corresponding to the projector plane location, wherein the second location corresponds to the projector plane location when a phase value of the second location is equal to a phase value of the projector plane location; and

computing a depth value for the projector location based on the second location.

10. The non-transitory computer readable medium of claim 9 , wherein mapping a location comprises:

mapping the projector plane location to a location in the TOF depth image using a projector-TOF sensor calibration matrix; and

mapping the location in the TOF depth image to the first location using a camera-TOF sensor calibration matrix.

11. The non-transitory computer readable medium of claim 9 , wherein the camera is the TOF sensor.

12. The non-transitory computer readable medium of claim 11 , wherein the projector is a digital micromirror device using a phase modulated light emitting diode as a light source.