Systems and methods for dynamic range compression in multi-frame processing

1. A method comprising:

obtaining, using at least one processor of an electronic device, a high dynamic range (HDR) input Bayer image;

generating, using the at least one processor of the electronic device, a plurality of synthesized images at different exposure levels based on the input Bayer image;

fusing, using the at least one processor of the electronic device, the synthesized images to generate a fused image;

generating, using the at least one processor of the electronic device, a gain map based on the fused image; and

applying, using the at least one processor of the electronic device, a gain based on the gain map to the input Bayer image;

wherein generating the gain map comprises:

transforming the fused image into a first RGB image; and

converting the input Bayer image into a second RGB image using a demosaic function to compute red, green, and blue plane gain maps, the gain map based on the first RGB image and the red, green, and blue plane gain maps.

2. The method of claim 1 , further comprising:

converting the input Bayer image into a Luma domain image;

wherein generating the plurality of synthesized images comprises:

generating a first lookup table (LUT) based on a compression LUT and a contrast LUT, the compression LUT and the contrast LUT based on the input Bayer image;

generating a second LUT based on the first LUT; and

applying the first LUT and the second LUT to the Luma domain image.

3. The method of claim 1 , wherein generating the plurality of synthesized images comprises generating YUV domain images based on the input Bayer image; and

wherein generating the YUV domain images comprises:

determining first, second, and third scales based on scene statistics;

generating first, second, and third exposure images based on the first, second, and third scales, respectively; and

clipping the first, second, and third exposure images into first, second, and third Bayer images, respectively.

4. The method of claim 3 , wherein generating the YUV domain images further comprises:

converting the first, second, and third Bayer images into third, fourth, and fifth RGB images, respectively;

applying first, second, and third gain values to the third, fourth, and fifth RGB images, respectively, to generate first, second, and third adjusted images;

performing first, second, and third color corrections based on the first, second, and third adjusted images, respectively, to generate first, second, and third color-corrected images;

compressing the first, second, and third color-corrected images to generate first, second, and third compressed color-corrected images; and

converting the first, second, and third compressed color-corrected images into first, second, and third YUV images, respectively.

5. The method of claim 3 , wherein determining the first, second, and third scales comprises at least one of:

increasing a scale value based on a low brightness level; and

decreasing a scale value based on a high brightness level without exceeding a threshold.

6. The method of claim 1 , further comprising:

determining a pixel-wise compression level based on the gain map; and

applying the pixel-wise compression level to the input Bayer image.

7. The method of claim 1 , further comprising:

outputting a displayable range image based on the applied gain, wherein the applied gain is determined as a weighted average of three color planes;

wherein a gain value applied to each sample in the displayable range image comprises an interpolated value based on the gain map.

8. An electronic device comprising:

at least one processing device configured to:

obtain a high dynamic range (HDR) input Bayer image;

generate a plurality of synthesized images at different exposure levels based on the input Bayer image;

fuse the synthesized images to generate a fused image;

generate a gain map based on the fused image;

apply a gain based on the gain map to the input Bayer image; and

output a displayable range image based on the applied gain, wherein the applied gain is determined as a weighted average of three color planes;

wherein a gain value applied to each sample in the displayable range image comprises an interpolated value based on the gain map.

9. The electronic device of claim 8 , wherein:

the at least one processing device is further configured to convert the input Bayer image into a Luma domain image; and

to generate the plurality of synthesized images, the at least one processing device is configured to:

generate a first lookup table (LUT) based on a compression LUT and a contrast LUT, the compression LUT and the contrast LUT based on the input Bayer image;

generate a second LUT based on the first LUT; and

apply the first LUT and the second LUT to the Luma domain image.

10. The electronic device of claim 8 , wherein:

to generate the plurality of synthesized images, the at least one processing device is configured to generate YUV domain images based on the input Bayer image; and

to generate the YUV domain images, the at least one processing device is configured to:

determine first, second, and third scales based on scene statistics;

generate first, second, and third exposure images based on the first, second, and third scales, respectively; and

clip the first, second, and third exposure images into first, second, and third Bayer images, respectively.

11. The electronic device of claim 10 , wherein, to generate the YUV domain images, the at least one processing device is further configured to:

convert the first, second, and third Bayer images into first, second, and third RGB images, respectively;

apply first, second, and third gain values to the first, second, and third RGB images, respectively, to generate first, second, and third adjusted images;

perform first, second, and third color corrections based on the first, second, and third adjusted images, respectively, to generate first, second, and third color-corrected images;

compress the first, second, and third color-corrected images to generate first, second, and third compressed color-corrected images; and

convert the first, second, and third compressed color-corrected images into first, second, and third YUV images, respectively.

12. The electronic device of claim 10 , wherein, to determine the first, second, and third scales, the at least one processing device is configured to at least one of:

increase a scale value based on a low brightness level; and

decrease a scale value based on a high brightness level without exceeding a threshold.

13. The electronic device of claim 8 , wherein, to generate the gain map, the at least one processing device is configured to:

transform the fused image into a first RGB image; and

convert the input Bayer image into a second RGB image using a demosaic function to compute red, green, and blue plane gain maps; and

wherein the gain map is based on the first RGB image and the red, green, and blue plane gain maps.

14. The electronic device of claim 8 , wherein the at least one processing device is further configured to:

determine a pixel-wise compression level based on the gain map; and

apply the pixel-wise compression level to the input Bayer image.

15. A non-transitory machine-readable medium containing instructions that when executed cause at least one processor of an electronic device to:

obtain a high dynamic range (HDR) input Bayer image;

convert the input Bayer image into a Luma domain image;

generate a plurality of synthesized images at different exposure levels based on the Luma domain image;

fuse the synthesized images to generate a fused image;

generate a gain map based on the fused image; and

apply a gain based on the gain map to the input Bayer image;

wherein the instructions that when executed cause the at least one processor to generate the plurality of synthesized images comprise instructions that when executed cause the at least one processor to:

generate a first lookup table (LUT) based on a compression LUT and a contrast LUT, the compression LUT and the contrast LUT based on the input Bayer image;

generate a second LUT based on the first LUT; and

apply the first LUT and the second LUT to the Luma domain image.

16. A non-transitory machine-readable medium containing instructions that when executed cause at least one processor of an electronic device to:

obtain a high dynamic range (HDR) input Bayer image;

generate a plurality of synthesized images at different exposure levels based on the input Bayer image;

fuse the synthesized images to generate a fused image;

generate a gain map based on the fused image; and

apply a gain based on the gain map to the input Bayer image;

wherein the instructions that when executed cause the at least one processor to generate the plurality of synthesized images comprise instructions that when executed cause the at least one processor to generate YUV domain images based on the input Bayer image; and

wherein the instructions that when executed cause the at least one processor to generate the YUV domain images comprise instructions that when executed cause the at least one processor to:

determine first, second, and third scales based on scene statistics;

generate first, second, and third exposure images based on the first, second, and third scales, respectively; and

clip the first, second, and third exposure images into first, second, and third Bayer images, respectively.

17. The non-transitory machine-readable medium of claim 16 , further containing instructions that when executed cause the at least one processor to convert the input Bayer image into a Luma domain image;

wherein the instructions that when executed cause the at least one processor to generate the plurality of synthesized images comprise instructions that when executed cause the at least one processor to:

generate a first lookup table (LUT) based on a compression LUT and a contrast LUT, the compression LUT and the contrast LUT based on the input Bayer image;

generate a second LUT based on the first LUT; and

apply the first LUT and the second LUT to the Luma domain image.

18. The non-transitory machine-readable medium of claim 16 , wherein the instructions that when executed cause the at least one processor to generate the YUV domain images further comprise instructions that when executed cause the at least one processor to:

convert the first, second, and third Bayer images into first, second, and third RGB images, respectively;

apply first, second, and third gain values to the first, second, and third RGB images, respectively, to generate first, second, and third adjusted images;

perform first, second, and third color corrections based on the first, second, and third adjusted images, respectively, to generate first, second, and third color-corrected images;

compress the first, second, and third color-corrected images to generate first, second, and third compressed color-corrected images; and

convert the first, second, and third compressed color-corrected images into first, second, and third YUV images, respectively.

19. The non-transitory machine-readable medium of claim 16 , wherein the instructions that when executed cause the at least one processor to determine the first, second, and third scales comprise instructions that when executed cause the at least one processor to at least one of:

increase a scale value based on a low brightness level; and

decrease a scale value based on a high brightness level without exceeding a threshold.

20. The non-transitory machine-readable medium of claim 15 , further containing instructions that when executed cause the at least one processor to output a displayable range image based on the applied gain, wherein the applied gain is determined as a weighted average of three color planes;

wherein a gain value applied to each sample in the displayable range image comprises an interpolated value based on the gain map.