Blending depth images obtained with multiple exposures
To extend dynamic range, a depth imager can expose a scene with two different exposures to obtain two depths. The two different exposures are selected to have respective depth ranges that overlap each other. The two depths can be combined by taking into account the difference in exposures and where the two depths are located with respect to boundaries of an overlapping range of the two depth ranges. The combination of the two depths in this manner can help avoid image artifacts such as contouring.
1. A method for extending dynamic range of a depth imager through multiple exposures, the method comprising:
determining a first depth obtained from a first exposure of a scene;
determining a second depth obtained from a second exposure of the scene, wherein the second exposure is different from the first exposure;
computing an estimated depth based on the first depth, the second depth, and a relationship between the first exposure and the second exposure; and
computing an adjusted depth based on the estimated depth, the relationship between the first exposure and the second exposure, a saturation distance of the second exposure, and a noise tolerance distance of the first exposure.
2. The method of claim 1 , wherein a first depth range corresponding to the first exposure overlaps with a second depth range corresponding to the second exposure.
3. The method of claim 1 , wherein:
the saturation distance of the second exposure is less than the noise tolerance distance of the first exposure.
4. The method of claim 1 , wherein:
the relationship between the first exposure and the second exposure is based on a ratio of the second exposure and the first exposure.
5. The method of claim 1 , wherein:
the relationship between the first exposure and the second exposure defines an exposure weight that weighs the second depth more than the first depth.
6. The method of claim 1 , wherein computing the estimated depth comprises:
determining a weighted combination of the first depth and the second depth based on the relationship between the first exposure and the second exposure.
7. The method of claim 1 , wherein computing the adjusted depth comprises:
in response to determining that the estimated depth is less than the saturation distance of the second exposure, setting the first depth as the adjusted depth.
8. The method of claim 1 , wherein computing the adjusted depth comprises:
in response to determining that the estimated depth is greater than the noise tolerance distance of the first exposure, setting the second depth as the adjusted depth.
9. The method of claim 1 , wherein computing the adjusted depth comprises:
determining a combined weight based on (1) the relationship between the first exposure and the second exposure, and (2) an interpolation weight defined by the estimated depth, the saturation distance of the second exposure, and the noise tolerance distance of the first exposure.
10. The method of claim 9 , wherein:
the interpolation weight depends on a proximity of the estimated depth to the noise tolerance distance of the first exposure or the saturation distance of the second exposure.
11. The method of claim 9 , wherein computing the adjusted depth comprises:
computing a weighted combination of the first depth and the second depth based on the combined weight; and
setting the weighted combination of the first depth and the second depth as the adjusted depth.
12. The method of claim 1 , further comprising:
in response to determining that the second depth indicates saturation, setting the first depth as the adjusted depth.
13. A depth imager with extended dynamic range, the depth imager comprising:
a time-of-flight controller to control a light source and one or more sensors to make a first measurement based on a first exposure of a scene and a second measurement based on a second exposure of the scene different from the first exposure;
a depth calculator to calculate a first depth based on the first measurement and a second depth based on the second measurement; and
a blender to:
compute an estimated depth based on the first depth and the second depth; and
output an adjusted depth based on the first depth, the second depth, an exposure weight relating the first exposure and the second exposure, and an interpolation weight computed based on the estimated depth.
14. The depth imager of claim 13 , wherein the time-of-flight controller controls the light source to emit a first number of light pulses for the first exposure, and controls the light source to emit a second number of light pulses for the second exposure.
15. The depth imager of claim 14 , wherein the exposure weight is based on a ratio of the second number and the first number.
16. The depth imager of claim 13 , wherein the blender computes the estimated depth by calculating an average of the first depth and the second depth weighted by the exposure weight.
17. The depth imager of claim 13 , wherein the blender outputs the first depth as the adjusted depth if the estimated depth is less than a saturation distance of the second exposure.
18. The depth imager of claim 13 , wherein the blender outputs the second depth as the adjusted depth if the estimated depth is greater than a noise tolerance distance of the first exposure.
19. The depth imager of claim 13 , wherein the blender outputs an average of the first depth and the second depth as the adjusted depth if the estimated depth is between a saturation distance of the second exposure and a noise tolerance distance of the first exposure, wherein the average is weighted by the exposure weight and the interpolation weight.
20. A depth imager with extended dynamic range, the depth imager comprising:
means for making a first measurement from exposing a scene with a first number of light pulses and a second measurement from exposing the scene with a second number of light pulses;
means for determining a first depth from the first measurement and a second depth from the second measurement;
means for determining an estimated depth based on the first depth and the second depth; and
means for outputting an adjusted depth by blending the first depth and the second depth with a combination of (1) an exposure weight relating the first number of light pulses and the second number of light pulses, and (2) an interpolation weight, wherein the interpolation weight is determined based on the estimated depth.
21. The depth imager of claim 20 , wherein the exposure weight is based on a ratio of the second number and the first number.
22. The depth imager of claim 20 , wherein the estimated depth is an average of the first depth and the second depth weighted by the exposure weight.
23. The depth imager of claim 20 , wherein the means for outputting the adjusted depth comprises:
means for outputting the first depth as the adjusted depth if the estimated depth is less than a saturation distance of the second measurement;
means for outputting the second depth as the adjusted depth if the estimated depth is greater than a noise tolerance distance of the first measurement; and
means for outputting an average of the first depth and the second depth as the adjusted depth if the estimated depth is between a saturation distance of the second measurement and a noise tolerance distance of the first measurement, wherein the average is weighted by the exposure weight and the interpolation weight.