Digital image exposure correction
View Patent ↗Techniques are disclosed for correcting the exposure of a digital image. An exposure predictor may be generated based on a set of images for which ground truth data are known. After identifying an optimal set of features, the exposure of the digital image may be corrected by extracting values of the selected optimal features from the image, using the predictor to predict a desired exposure correction for the image, and correcting the exposure of the image by the predicted desired amount. Exposure correction is based on a model that relates intensity of light in the world to the RGB digits of the digital image. The model comprises a gamma function that models the response of a typical monitor and a S-shaped curve that compresses the large dynamic range of the world to the small dynamic range of the RGB digit space.
1. A method for processing an image, the method comprising:
determining an absolute logarithm of an average luminance ratio for each region of a plurality of predetermined regions within the image, wherein the average luminance ratio of a region comprises a ratio of the average luminance of the region to an average luminance of the image excluding the region;
identifying a region, from among the plurality of predetermined regions, with the highest absolute logarithm of the average luminance ratio of the region, the region with the highest absolute logarithm of the average luminance ratio being an active region of interest;
extracting, from the image, values of at least one feature derived from the active region of interest in the image;
predicting a desired exposure correction of the image based on the extracted values; and
correcting, via a processor, the exposure of the image by the predicted exposure correction to produce an exposure-corrected image;
wherein the active region of interest has a base size that is proportional to the dimensions of the image, wherein the dimensions of the active region of interest are proportional to the base size multiplied by a measure of average activity in the image, and wherein the average activity is based, at least in part, on luminance within the image.
2. The method of claim 1 , wherein the active region comprises a plurality of pixels satisfying an activity threshold.
3. The method of claim 2 , wherein an average activity within the active region is above a predetermined minimum activity threshold.
4. The method of claim 1 , wherein the extracted feature values are based on values of a luminance channel of the image.
5. A method for processing an image, the method comprising:
extracting, from the image, values of at least one feature derived from an active region of interest in the image, wherein the extracting comprises:
identifying the active region of interest in the image,
screening the image for activity to generate an active image,
generating an average histogram as a weighted average of a histogram of the active region of interest and a histogram of the active image, and
extracting values of features from the average histogram;
predicting a desired exposure correction of the image based on the extracted values; and
correcting, via a processor, the exposure of the image by the predicted exposure correction to produce an exposure-corrected image;
wherein the average histogram, the histogram of the active region of interest, and the histogram of the active image each represent a count of image sectors as a function of luminance of the image sectors.
6. The method of claim 5 , wherein said generating an average histogram as a weighted average of a histogram of the active region of interest and a histogram of the active image comprises:
calculating a likelihood that the active region of interest influences the desired exposure shift of the image; and
generating the average histogram as a average of the histogram of the active region of interest and a histogram of the active image, weighted by the likelihood.
7. The method of claim 6 , wherein said calculating a likelihood that the active region of interest influences the desired exposure shift of the image comprises calculating the likelihood using the formula:
p
(
ROI
)
=
1
1
+
exp
(
-
s
(
log
2
r
ROI
|
-
o
)
)
wherein o represents the luminance difference in stops between the active region of interest and that portion of the image not including the active region of interest when the likelihood is equal to 0.5, wherein s is proportional to the slope of the formula at |log 2 r ROI |=O, and wherein r ROI is the ratio of the average linear luminance in the active region of interest to the average linear luminance of that portion of the image not including the active region of interest.
8. The method of claim 5 , wherein said extracting values of features from the average histogram comprises extracting values of linear features from the average histogram.
9. The method of claim 8 , wherein said extracting values of features from the average histogram comprises extracting at least one moment from the average histogram.
10. The method of claim 5 , wherein said extracting values of features from the average histogram comprises extracting values of nonlinear features from the average histogram.
11. The method of claim 10 , wherein said extracting values of features from the average histogram comprises extracting at least one percentile from the average histogram.
12. The method of claim 5 , wherein the image sectors each consist of a single pixel.
13. The method of claim 5 , wherein the image sectors each consist of a plurality of pixels.
14. An apparatus for processing an image, the apparatus comprising:
means for determining an absolute logarithm of an average luminance ratio for each region of a plurality of predetermined regions within the image, wherein the average luminance ratio of a region comprises a ratio of the average luminance of the region to an average luminance of the image excluding the region;
means for identifying the region, from among the plurality of predetermined regions, with the highest absolute logarithm of the average luminance ratio of the region, the region with the highest absolute logarithm of the average luminance ratio being an active region of interest;
means for extracting, from the image, values of at least one feature derived from the active region of interest in the image;
means for predicting a desired exposure correction of the image based on the extracted feature values; and
means for correcting the exposure of the image by the predicted exposure correction to produce an exposure-corrected image;
wherein the active region of interest has a base size that is proportional to the dimensions of the image, wherein the dimensions of the active region of interest are proportional to the base size multiplied by a measure of average activity in the image, and wherein the average activity is based, at least in part, on luminance within the image.
15. An apparatus for processing an image, the apparatus comprising:
means for determining an absolute logarithm of an average luminance ratio for each region of a plurality of predetermined regions within the image, wherein the average luminance ratio of a region comprises a ratio of the average luminance of the region to an average luminance of the image excluding the region;
means for identifying the region, from among the plurality of predetermined regions, with the highest absolute logarithm of the average luminance ratio of the region, the region with the highest absolute logarithm of the average luminance ratio being an active region of interest;
means for extracting, from the image, values of at least one feature derived from the active region of interest in the image;
means for predicting a desired exposure correction of the image based on the extracted feature values; and
means for correcting the exposure of the image by the predicted exposure correction to produce an exposure-corrected image;
wherein the means for extracting comprises:
means for identifying the active region of interest in the image;
means for screening the image for activity to generate an active image;
means for generating an average histogram as a weighted average of a histogram of the active region of interest and a histogram of the active image; and
means for extracting values of features from the average histogram;
wherein the average histogram, the histogram of the active region of interest, and the histogram of the active image each represent a count of image sectors as a function of luminance of the image sectors.
16. The apparatus of claim 15 , wherein the means for generating comprises:
means for calculating a likelihood that the active region of interest influences the desired exposure shift of the image; and
means for generating the average histogram as a average of the histogram of the active region of interest and a histogram of the active, weighted by the likelihood.
17. The apparatus of claim 16 , wherein the means for calculating comprises means for calculating the likelihood using the formula:
p
(
ROI
)
=
1
1
+
exp
(
-
s
(
log
2
r
ROI
|
-
o
)
)
wherein o represents the luminance difference in stops between the active region of interest and that portion of the image not including the active region of interest when the likelihood is equal to 0.5, wherein s is proportional to the slope of the formula at |log 2 r ROI |=o, and wherein r ROI is the ratio of the average linear luminance in the active region of interest to the average linear luminance of that portion of the image not including the active region of interest.
18. The apparatus of claim 15 , wherein the means for generating the average histogram comprises means for extracting values of linear features from the average histogram.
19. The apparatus of claim 18 , wherein the means for generating the average histogram comprises means for extracting at least one moment from the average histogram.
20. The apparatus of claim 15 , wherein the means for generating the average histogram comprises means for extracting values of nonlinear features from the average histogram.
21. The apparatus of claim 20 , wherein the means for generating the average histogram comprises means for extracting at least one percentile from the average histogram.
22. The apparatus of claim 15 , wherein the image sectors each consist of a single pixel.
23. The apparatus of claim 15 , wherein the image sectors each consist of a plurality of pixels.
24. A computer-readable storage medium comprising instructions for processing an image, the instructions comprising:
instructions for determining an absolute logarithm of an average luminance ratio for each region of a plurality of predetermined regions within the image, wherein the average luminance ratio of a region comprises a ratio of the average luminance of the region to an average luminance of the image excluding the region;
instructions for identifying the region, from among the plurality of predetermined regions, with the highest absolute logarithm of the average luminance ratio of the region, the region with the highest absolute logarithm of the average luminance ratio being an active region of interest;
instructions for extracting, from the image, values of at least one feature derived from the active region of interest in the image;
instructions for predicting a desired exposure correction of the image based on the extracted feature values; and
instructions for correcting the exposure of the image by the predicted exposure correction to produce an exposure-corrected image;
wherein the active region of interest has a base size that is proportional to the dimensions of the image, wherein the dimensions of the active region of interest are proportional to the base size multiplied by a measure of average activity in the image, and wherein the average activity is based, at least in part, on luminance within the image.
25. A computer-readable storage medium comprising instructions for processing an image, the instructions comprising:
instructions for extracting, from the image, values of at least one feature derived from an active region of interest in the image, and wherein the feature extractor is further configured to:
instructions for identifying the active region of interest in the image,
instructions for screening the image for activity to generate an active image,
instructions for generating an average histogram as a weighted average of a histogram of the active region of interest and a histogram of the active image, and
instructions for extracting values of features from the average histogram;
instructions for predicting a desired exposure correction of the image based on the extracted feature values; and
instructions for correcting the exposure of the image by the predicted exposure correction to produce an exposure-corrected image;
wherein the average histogram, the histogram of the active region of interest, and the histogram of the active image each represent a count of image sectors as a function of luminance of the image sectors.
26. The computer-readable storage medium of claim 25 , wherein the image sectors each consist of a single pixel.
27. The computer-readable storage medium of claim 25 , wherein the image sectors each consist of a plurality of pixels.