Representing and reconstructing high dynamic range images
A high dynamic range image can be recovered from a full-resolution lower-dynamic-range image and a reduced-resolution higher-dynamic-range image. Information regarding higher spatial frequencies may be obtained by extracting high spatial frequencies from the lower-dynamic-range image. In some embodiments an approximate impulse-response function is determined by comparing the higher- and lower-dynamic range images. A scaling image obtained by applying the impulse-response function to a high-frequency band of the lower-dynamic range image is combined with an upsampled higher-dynamic range image to yield a reconstructed image.
1. Apparatus for reconstructing a higher-dynamic-range image from a full resolution, lower dynamic range (FRLDR) representation of an image and a lower resolution, higher-dynamic-range (LRHDR) representation of the image, the LRHDR representation having a lower resolution and a higher dynamic range than the FRLDR representation, the apparatus comprising:
a spatial filtering unit configured to generate a scaling image comprising information relating to high spatial frequency components of the FRLDR representation;
an upsampling unit configured to generate an upsampled image from the LRHDR representation; and,
a combining unit configured to combine the scaling image and the upsampled image to obtain a reconstructed higher-dynamic-range image,
wherein each of the spatial filtering unit, upsampling unit and combining unit comprises image-processing hardware, software stored on a non-transitory computer-readable medium and executable by a data processor, or a combination thereof.
2. Apparatus according to claim 1 comprising a display wherein the apparatus is configured to display the reconstructed higher-dynamic range image on the display.
3. Apparatus according to claim 1 wherein the FRLDR representation is created from an original full resolution, high-dynamic-range representation of the image using a tone-mapping operation and wherein the spatial filtering unit is configured to perform an inverse tone-mapping operation.
4. Apparatus according to claim 1 wherein the spatial filtering unit is configured to:
downsample the FRLDR representation to obtain a downsampled image having a resolution sufficiently low to comprise only spatial frequency components below a desired spatial frequency level;
upsample the downsampled image to obtain a resampled image; and
combine the resampled image and the FRLDR representation to generate the scaling image.
5. Apparatus according to claim 4 wherein the spatial filtering unit is configured to divide each pixel value of the FRLDR representation by a corresponding pixel value of the resampled image.
6. Apparatus according to claim 4 wherein an average of pixel values in the scaling image is 1.
7. Apparatus according to claim 1 wherein the combining unit is configured to multiply each pixel value of the scaling image with a corresponding pixel value of the upsampled image.
8. Apparatus according to claim 1 wherein the spatial filtering unit is configured to apply a convolution filter to the FRLDR representation.
9. Apparatus according to claim 1 wherein the spatial filtering unit is configured to convert a color space of the FRLDR representation into a color space of the LRHDR representation.
10. Apparatus according to claim 1 wherein the spatial filtering unit is configured to:
determine an impulse-response function from the FRLDR representation and the LRHDR representation;
extract high spatial frequency components of the FRLDR representation; and
multiply each pixel value of the extracted high spatial frequency components by a corresponding value of the impulse-response function to generate the scaling image.
11. Apparatus according to claim 10 wherein the spatial filtering unit is configured to apply a quantization threshold to the high spatial frequency components.
12. Apparatus according to claim 11 wherein the spatial filtering unit is configured to:
extract high spatial frequency components of the LRHDR representation to obtain a high frequency band of the LRHDR representation;
reduce a resolution of the FRLDR representation to obtain a reduced resolution low-dynamic-range image;
extract high spatial frequency components of the reduced resolution low-dynamic-range image to obtain a high frequency band of the reduced resolution low-dynamic-range image; and
process the high frequency band of the LRHDR representation and the high frequency band of the reduced resolution low-dynamic-range image to determine the impulse-response function.
13. Apparatus according to claim 12 wherein the spatial filtering unit is configured to:
select a group of pixels from within a spatial region of the high frequency band of the LRHDR representation and selecting a corresponding group of pixels from within the spatial region of the high frequency band of the reduced resolution low-dynamic-range image;
sort the group of pixels and the corresponding group of pixels by pixel value to obtain a pair of sorted arrays defining a regional impulse-response function corresponding to the spatial region; and,
combine one or more regional impulse-response functions to obtain the impulse-response function.
14. Apparatus according to claim 13 wherein the spatial filtering unit is configured to select and sort groups of pixels for a plurality of spatial regions.
15. Apparatus according to claim 13 wherein the spatial filtering unit is configured to:
select a plurality of pairs of pixel values from the pair of sorted arrays, each pair of pixel values comprising pixel values having a common index within the pair of sorted arrays; and,
interpolate between the plurality of pairs of pixel values.
16. Apparatus according to claim 13 wherein the spatial filtering unit is configured to:
determine a regional impulse-response function value for each of a plurality of regions that include the pixel; and,
computing a weighted combination of the regional impulse-response function values.
17. Apparatus according to claim 16 wherein the spatial filtering unit is configured to weight the regional impulse-response function values based on distances of the pixel from central points in the regions to which the regional impulse-response function values correspond.
18. Apparatus according to claim 15 wherein the spatial filtering unit is configured to extrapolate the regional impulse-response function past at least one of the smallest and largest ones of the pairs of pixel values.
19. A non-transitory computer-readable medium with a set of instructions stored thereon, that, when loaded into an electronic controller, cause the controller to perform the steps of:
receiving a full resolution, lower dynamic range (FRLDR) representation of the image and a lower resolution, higher-dynamic-range (LRHDR) representation of the image, the LRHDR representation having a lower resolution and a higher dynamic range than the FRLDR representation;
processing at least the FRLDR representation to obtain a scaling image comprising information relating to high spatial frequency components of the FRLDR representation;
upsampling the LRHDR representation to obtain an upsampled image; and
combining the scaling image and the upsampled image to obtain a reconstructed higher-dynamic-range image.
20. Apparatus according to claim 1 wherein the apparatus is integrated with a DVD player.
21. Apparatus according to claim 1 wherein the apparatus is integrated with a video player.
22. Apparatus according to claim 1 wherein the apparatus is integrated with a computer display, the apparatus configured to display the reconstructed higher-dynamic range image on the computer display.
23. Apparatus according to claim 1 wherein the apparatus is integrated with a television having a display, the apparatus configured to display the reconstructed higher-dynamic range image on the display.
24. Apparatus according to claim 1 wherein the apparatus is integrated with an image-displaying device having a display, the apparatus configured to display the reconstructed higher-dynamic range image on the display.