Representing and reconstructing high dynamic range images
A high dynamic range image can be recovered from a full-resolution lower-dynamic-range image and a reduced-resolution higher-dynamic-range image. Information regarding higher spatial frequencies may be obtained by extracting high spatial frequencies from the lower-dynamic-range image. In some embodiments an approximate impulse-response function is determined by comparing the higher- and lower-dynamic range images. A scaling image obtained by applying the impulse-response function to a high-frequency band of the lower-dynamic range image is combined with an upsampled higher-dynamic range image to yield a reconstructed image.
1. A computer-implemented method for reconstructing a higher-dynamic-range image, the method comprising:
obtaining a full resolution, lower dynamic range (FRLDR) representation of the image and a lower resolution, higher-dynamic-range (LRHDR) representation of the image, the LRHDR representation having a lower resolution and a higher dynamic range than the FRLDR representation; and
in a data processor:
processing at least the FRLDR representation to obtain a scaling image comprising information relating to high spatial frequency components of the FRLDR representation;
upsampling the LRHDR representation to obtain an upsampled image; and
combining the scaling image and the upsampled image to obtain a reconstructed higher-dynamic-range image.
2. A method according to claim 1 wherein the FRLDR representation is created from an original full resolution, high-dynamic-range representation of the image using a tone-mapping operation and wherein processing at least the FRLDR representation comprises inverting the tone-mapping operation.
3. A method according to claim 1 wherein processing at least the FRLDR representation to obtain the scaling image comprises:
downsampling the FRLDR representation to obtain a downsampled image wherein the downsampled image has a resolution sufficiently low to comprise only spatial frequency components below a desired spatial frequency level;
upsampling the downsampled image to obtain a resampled image; and
combining the resampled image and the FRLDR representation to obtain the scaling image.
4. A method according to claim 3 wherein combining the resampled image and the FRLDR representation comprises dividing each pixel value of the FRLDR representation by a corresponding pixel value of the resampled image.
5. A method according to claim 3 wherein an average of pixel values in the scaling image is 1.
6. A method according to claim 1 wherein combining the scaling image and the upsampled image comprises multiplying each pixel value of the scaling image with a corresponding pixel value of the upsampled image.
7. A method according to claim 1 wherein processing at least the FRLDR representation to obtain the scaling image comprises applying a convolution filter to the FRLDR representation.
8. A method according to claim 1 wherein processing the FRLDR representation comprises converting a color space of the FRLDR representation into a color space of the LRHDR representation.
9. A method according to claim 1 wherein processing at least the FRLDR representation to obtain the scaling image comprises:
using the FRLDR representation and the LRHDR representation to determine an impulse-response function;
extracting high spatial frequency components of the FRLDR representation; and
multiplying each pixel value of the extracted high spatial frequency components by a corresponding value of the impulse-response function to determine the scaling image.
10. A method according to claim 9 comprising applying a quantization threshold to the high spatial frequency components.
11. A method according to claim 10 wherein using the FRLDR representation and the LRHDR representation to determine an impulse-response function comprises:
extracting high spatial frequency components of the LRHDR representation to obtain a high frequency band of the LRHDR representation;
reducing a resolution of the FRLDR representation to obtain a reduced resolution low-dynamic-range image;
extracting high spatial frequency components of the reduced resolution low-dynamic-range image to obtain a high frequency band of the reduced resolution low-dynamic-range image; and
processing the high frequency band of the LRHDR representation and the high frequency band of the reduced resolution low-dynamic-range image to obtain the impulse-response function.
12. A method according to claim 11 wherein processing the high frequency band of the LRHDR representation and the high frequency band of the reduced resolution low-dynamic-range image to obtain the impulse-response function comprises:
(a) selecting a group of pixels from within a spatial region of the high frequency band of the LRHDR representation and selecting a corresponding group of pixels from within the spatial region of the high frequency band of the reduced resolution low-dynamic-range image;
(b) sorting the group of pixels and the corresponding group of pixels by pixel value to obtain a pair of sorted arrays defining a regional impulse-response function corresponding to the spatial region;
(c) combining one or more regional impulse-response functions to obtain the impulse-response function.
13. A method according to claim 12 wherein combining one or more regional impulse-response functions comprises repeating steps (a) through (b) for a plurality of spatial regions.
14. A method according to claim 12 wherein step (b) comprises selecting a plurality of pairs of pixel values from the pair of sorted arrays, each pair of pixel values comprising pixel values having a common index within the pair of sorted arrays; and,
interpolating between the plurality of pairs of pixel values.
15. A method according to claim 12 comprising determining an impulse-response function value for a pixel by determining a regional impulse-response function value for each of a plurality of regions that include the pixel and computing a weighted combination of the regional impulse-response function values.
16. A method according to claim 15 comprising computing the weighted combination of the regional impulse-response function values comprises weighting the regional impulse-response function values based on distances of the pixel from central points in the regions to which the regional impulse-response function values correspond.
17. A method according to claim 14 wherein step (b) comprises extrapolating the regional impulse-response function past at least one of the smallest and largest ones of the pairs of pixel values.
18. Apparatus for reconstructing a higher-dynamic-range image, the apparatus comprising:
a data processor;
a spatial filtering stage configured to process at least a full resolution, lower dynamic range (FRLDR) representation of an image to obtain a scaling image comprising information relating to high spatial frequency components of the FRLDR representation;
an upsampling stage configured to upsample a lower resolution, higher-dynamic-range (LRHDR) representation of the image, to obtain an upsampled image; the LRHDR representation having a lower resolution and a higher dynamic range than the FRLDR representation;
and a combining stage configured to combine the scaling image and the upsampled image to obtain a reconstructed higher-dynamic-range image, wherein the spatial filtering stage, upsampling stage and combining stage each comprise computer-readable software instructions that are executable by the data processor, the computer-readable instructions stored in a program product comprising a tangible computer-readable medium.
19. Apparatus according to claim 18 comprising a display wherein the apparatus is configured to display the reconstructed higher-dynamic range image on the display.