Merging multiple exposures to generate a high dynamic range image
A method of generating a high dynamic range (HDR) image is provided that includes capturing a long exposure image and a short exposure image of a scene, computing a merging weight for each pixel location of the long exposure image based on a pixel value of the pixel location and a saturation threshold, and computing a pixel value for each pixel location of the HDR image as a weighted sum of corresponding pixel values in the long exposure image and the short exposure image, wherein a weight applied to a pixel value of the pixel location of the short exposure image and a weight applied to a pixel value of the pixel location in the pixel long exposure image are determined based on the merging weight computed for the pixel location and responsive to motion in a scene of the long exposure image and the short exposure image.
1. A method comprising:
receiving, by a gain adjust component, a long exposure image having a long exposure time;
receiving, by the gain adjust component, a short exposure image having a short exposure time;
determining, by the gain adjust component, a gain adjustment for the long exposure time based on the short exposure time divided by the long exposure time;
outputting, by the gain adjust component, an adjusted long exposure image based on the gain adjustment for the long exposure time multiplied with each pixel in the long exposure image;
outputting, by the gain adjust component, an adjusted short exposure image;
determining, by a weight component, a merging weight associated with the long exposure image by comparing a single pixel value of the long exposure image with a saturation threshold, wherein the merging weight is not determined based on a block of pixels around the single pixel value;
determining, by the weight component, a motion adaptation weight of the long exposure image based on a difference of luminance of the single pixel value of the long exposure image and a corresponding pixel value of the short exposure image;
determining, by the weight component, a first weight based on the merging weight and further based on the motion adaption weight;
receiving, by a merge component, the first weight, the adjusted long exposure image, and the adjusted short exposure image; and
outputting, by the merge component, a merged high dynamic range image based on the first weight, the adjusted long exposure image, and the adjusted short exposure image.
2. The method of claim 1 , wherein:
the long exposure time of the long exposure image and the short exposure time of the short exposure image are set according to a ratio.
3. The method of claim 2 , wherein:
the ratio is a power of 2.
4. The method of claim 1 , further comprising:
determining, by the weight component, a second weight by subtracting the first weight from one; and
receiving, by the merge component, the second weight,
wherein outputting the merged high dynamic range image is based on the first weight, the second weight, the adjusted long exposure image, and the adjusted short exposure image.
5. The method of claim 4 , wherein:
the merged high dynamic range image is determined, for each pixel, by:
multiplying the adjusted long exposure image with the first weight to determine a first value;
multiplying the adjusted short exposure image with the second weight to determine a second value; and
adding the first value with the second value.
6. The method of claim 1 ,
wherein the merging weight is a first per-pixel weight, and
wherein the first weight is a second per-pixel weight.
7. The method of claim 1 , wherein the motion adaption weight decreases when the difference of luminance increases at least in a part of a range of the difference of luminance.
8. An apparatus comprising:
a processor; and
a non-transitory computer readable storage medium storing a program for execution by the processor, the program including instructions to cause the processor to:
receive a long exposure image having a long exposure time;
receive a short exposure image having a short exposure time;
determine a gain adjustment for the long exposure time based on the short exposure time divided by the long exposure time;
determine an adjusted long exposure image based on the gain adjustment for the long exposure time multiplied with each pixel in the long exposure image
determine a merging weight associated with the long exposure image by comparing a single pixel value of the long exposure image with a saturation threshold, wherein the merging weight is not determined based on a block of pixels around the single pixel value;
determine a motion adaptation weight of the long exposure image based on a difference of luminance of the single pixel value of the long exposure image and a corresponding pixel value of the short exposure image;
determine a first weight based on the merging weight and further based on the motion adaption weight;
determine a merged high dynamic range image based on the first weight, the adjusted long exposure image, and an adjusted short exposure image; and
output the merged high dynamic range image.
9. The apparatus of claim 8 , wherein:
the long exposure time of the long exposure image and the short exposure time of the short exposure image are set according to a ratio.
10. The apparatus of claim 9 , wherein:
the ratio is a power of 2.
11. The apparatus of claim 8 , further comprising instructions to cause the processor to:
determine a second weight by subtracting the first weight from one; and
receive the second weight,
wherein the merged high dynamic range image is output based on the first weight, the second weight, the adjusted long exposure image, and the adjusted short exposure image.
12. The apparatus of claim 11 , wherein:
the merged high dynamic range image is determined, for each pixel, by:
multiplying the adjusted long exposure image with the first weight to determine a first value;
multiplying the adjusted short exposure image with the second weight to determine a second value; and
adding the first value with the second value.
13. The apparatus of claim 8 ,
wherein the merging weight is a first per-pixel weight, and
wherein the first weight is a second per-pixel weight.
14. The apparatus of claim 8 , wherein the motion adaption weight decreases when the difference of luminance increases at least in a part of a range of the difference of luminance.
15. A non-transitory computer-readable storage medium storing a program for execution by one or more processors, instructions to cause the one or more processors to:
receive a long exposure image having a long exposure time;
receive a short exposure image having a short exposure time;
determine a gain adjustment for the long exposure time based on the short exposure time divided by the long exposure time;
determine an adjusted long exposure image based on the gain adjustment for the long exposure time multiplied with each pixel in the long exposure image:
determine a merging weight associated with the long exposure image by comparing a single pixel value of the long exposure image with a saturation threshold, wherein the merging weight is not determined based on a block of pixels around the single pixel value;
determine a motion adaptation weight of the long exposure image based on a difference of luminance of the single pixel value of the long exposure image and a corresponding pixel value of the short exposure image;
determine a first weight based on the merging weight and further based on the motion adaption weight;
determine a merged high dynamic range image based on the first weight, the adjusted long exposure image, and an adjusted short exposure image; and
output the merged high dynamic range image.
16. The non-transitory computer-readable storage medium of claim 15 , wherein:
the long exposure time of the long exposure image and the short exposure time of the short exposure image are set according to a ratio having a power of 2.
17. The non-transitory computer-readable storage medium of claim 15 , further comprising instructions to cause the one or more processors to:
determine a second weight by subtracting the first weight from one; and
receive the second weight,
wherein the merged high dynamic range image is output based on the first weight, the second weight, the adjusted long exposure image, and the adjusted short exposure image.
18. The non-transitory computer-readable storage medium of claim 17 , wherein:
the merged high dynamic range image is determined, for each pixel, by:
multiplying the adjusted long exposure image with the first weight to determine a first value;
multiplying the adjusted short exposure image with the second weight to determine a second value; and
adding the first value with the second value.
19. The non-transitory computer-readable storage medium of claim 15 ,
wherein the merging weight is a first per-pixel weight, and
wherein the first weight is a second per-pixel weight.
20. The non-transitory computer-readable storage medium of claim 15 , wherein the motion adaption weight decreases when the difference of luminance increases at least in a part of a range of the difference of luminance.