Dynamic calibration correction in multi-frame, multi-exposure capture
A method for dynamic calibration correction in multi-frame, multi-exposure capture in a camera system includes receiving an amplified non-reference frame (e.g., with non-standard exposure value EV-4) to which a digital gain was applied and a reference frame (e.g., with standard exposure value EV0). The method includes generating a mask based on the reference frame and the amplified non-reference frame. The method includes estimating a scaling coefficient and an offset coefficient for each of a number of channels in the amplified non-reference frame, based on selected pixel locations in the generated mask. The method includes correcting a calibration of the amplified non-reference frame by applying the scaling coefficients for each of the channels. The method also includes outputting a calibration-corrected amplified non-reference frame to a multi-frame processor.
1. A method for dynamic calibration correction, the method comprising:
receiving an amplified non-reference frame to which a digital gain was applied and a reference frame;
generating a mask based on the reference frame and the amplified non-reference frame;
estimating a scaling coefficient and an offset coefficient for each of a number of channels in the amplified non-reference frame based on a selected pixel location in the generated mask;
correcting a calibration of the amplified non-reference frame by applying the scaling and offset coefficients for each of the channels; and
outputting a calibration-corrected amplified non-reference frame to a multi-frame processor.
2. The method of claim 1 , wherein the number of channels are one of:
R, G, and B channels corresponding to an RGB domain; or
a number of channels of a Bayer domain.
3. The method of claim 1 , wherein the correcting of the calibration of the amplified non-reference frame comprises one of:
a linear calibration correction; or
a non-linear calibration correction.
4. The method of claim 1 , wherein the generating of the mask comprises:
computing a non-reference mask based on luma of the amplified non-reference frame; and
computing a reference mask based on luma of the reference frame.
5. The method of claim 4 , wherein:
the computing of the non-reference mask is based on a comparison of a non-reference luma threshold to the luma of the amplified non-reference frame; and
the computing of the reference mask is based on a comparison of a reference luma threshold to the luma of the reference frame.
6. The method of claim 4 , further comprising:
computing the luma of the amplified non-reference frame by averaging R, G, and B channels of the amplified non-reference frame; and
computing the luma of the reference frame by averaging R, G, and B channels of the reference frame.
7. The method of claim 1 , wherein the estimating of the scaling coefficient and the offset coefficient is based on a specified objective defined as
min
a
,
b
∑
r
,
n
(
r
-
a
·
n
-
b
)
2
.
8. The method of claim 1 , further comprising:
preserving a saturation area of the amplified non-reference frame by:
detecting the saturation area of the amplified non-reference frame; and
blending the detected saturation area into the calibration-corrected amplified non-reference frame.
9. The method of claim 8 , further comprising:
in response to detecting lens-shading corrected (LSC) data in at least one of the received reference frame or the received amplified non-reference frame, removing the LSC data from the at least one of the received reference frame or the received amplified non-reference frame;
generating the mask based on the reference frame and the amplified non-reference frame that no longer contain the LSC data; and
after the correcting of the calibration and before the blending, re-applying the LSC data to the calibration-corrected non-reference frame.
10. The method of claim 8 , wherein:
the amplified non-reference frame is a result of the digital gain applied to an original non-reference frame;
the detecting of the saturation area comprises generating a saturation map by suppressing a highest value among R, G, and B values at each pixel in the amplified non-reference frame based on a saturation threshold and a saturation slope; and
the blending comprises blending the amplified non-reference frame and the calibration-corrected amplified non-reference frame using the saturation map.
11. An electronic device comprising:
a processor coupled to a memory;
the memory storing instructions that, when executed by the processor, cause the processor to:
receive an amplified non-reference frame to which a digital gain was applied and a reference frame;
generate a mask based on the reference frame and the amplified non-reference frame;
estimate a scaling coefficient and an offset coefficient for each of a number of channels in the amplified non-reference frame based on a selected pixel location in the generated mask;
correct a calibration of the amplified non-reference frame by applying the scaling and offset coefficients for each of the channels; and
output a calibration-corrected amplified non-reference frame to a multi-frame processor.
12. The electronic device of claim 11 , wherein the number of channels are one of:
R, G, and B channels corresponding to an RGB domain; or a number of channels of a Bayer domain.
13. The electronic device of claim 11 , wherein the instructions cause the processor to correct the calibration of the amplified non-reference frame by one of:
a linear calibration correction; or
a non-linear calibration correction.
14. The electronic device of claim 11 , wherein the instructions cause the processor to generate the mask by:
computing a non-reference mask based on luma of the amplified non-reference frame; and
computing a reference mask based on luma of the reference frame.
15. The electronic device of claim 14 , wherein:
the computing of the non-reference mask is based on a comparison of a non-reference luma threshold to the luma of the amplified non-reference frame; and
the computing of the reference mask is based on a comparison of a reference luma threshold to the luma of the reference frame.
16. The electronic device of claim 14 , wherein the instructions cause the processor to:
compute the luma of the amplified non-reference frame by averaging R, G, and B channels of the amplified non-reference frame; and
compute the luma of the reference frame by averaging R, G, and B channels of the reference frame.
17. The electronic device of claim 11 , wherein the instructions cause the processor to estimate the scaling coefficient and the offset coefficient based on a specified objective defined as
min
a
,
b
∑
r
,
n
(
r
-
a
·
n
-
b
)
2
.
18. The electronic device of claim 11 , wherein the instructions cause the processor to:
preserve a saturation area of the amplified non-reference frame by:
detecting the saturation area of the amplified non-reference frame; and
blending the detected saturation area into the calibration-corrected amplified non-reference frame.
19. The electronic device of claim 18 , wherein the instructions cause the processor to:
in response to detecting lens-shading corrected (LSC) data in at least one of the received reference frame or the received amplified non-reference frame, remove the LSC data from the at least one of the received reference frame or the received amplified non-reference frame;
generate the mask based on the reference frame and the amplified non-reference frame that no longer contain the LSC data; and
after the correcting of the calibration and before the blending, re-apply the LSC data to the calibration-corrected non-reference frame.
20. The electronic device of claim 18 , wherein:
the amplified non-reference frame is a result of the digital gain applied to an original non-reference frame;
the detecting of the saturation area comprises generating a saturation map by suppressing a highest value among R, G, and B values at each pixel in the amplified non-reference frame based on a saturation threshold and a saturation slope; and
the blending comprises blending the amplified non-reference frame and the calibration-corrected amplified non-reference frame using the saturation map.