White balance adjustment device, image capture device, white balance adjustment method, storage medium, and integrated circuit
View Patent ↗A natural white balance is achieved in images that are captured while emitting a flash. The white balance of an image is adjusted using a WB adjustment portion and a mixture ratio calculation portion estimating a mixture ratio of an external light component and a flashed light component that are present in an image captured with emitting a flash, from the image obtained that is captured while emitting a flash and an image signal that is obtained without emitting a flash. Further, an external light WB coefficient determination portion determines an WB coefficient for the external light, a flashed light WB coefficient setting portion sets a WB coefficient for the flashed light, and a WB processing portion continuously performs WB processing on the image captured while emitting a flash by using the mixture ratio as an interpolation ratio.
1. A white balance adjustment device, comprising:
a mixture ratio calculation portion operable to calculate a mixture ratio of an external light component and a flashed light component that are present in a second image signal, the mixture ratio being calculated based on a first image signal that is obtained without irradiating a flashed light on an object to be captured, and the second image signal that is obtained by illuminating the flashed light on the object to be captured;
an external light WB coefficient determination portion operable to determine an external light WB coefficient, which is a white balance coefficient for the external light, the external light WB coefficient being determined from the first image signal;
a flashed light WB coefficient setting portion operable to set a flashed light WB coefficient, which is a white balance coefficient of the flashed light; and
a WB processing portion operable to continuously perform white balance adjustment of the second image signal between the external light WB coefficient and the flashed light WB coefficient by using the calculated mixture ratio as an interpolation ratio,
wherein the WB processing portion obtains a third image signal by performing white balance correction on the second image signal based on the external light WB coefficient,
wherein the WB processing portion obtains a fourth image signal by performing white balance correction on the second image signal based on the flashed light WB coefficient, and
wherein the WB processing portion obtains a final image signal, which takes on a continuous value, by interpolating a signal value of the third image signal and a signal value of the fourth image signal based on the calculated mixture ratio.
2. The white balance adjustment device according to claim 1 ,
wherein the mixture ratio calculation portion calculates an external light ratio, as the mixture ratio, according to the following formula:
(the external light ratio)=(a signal value of the first image signal)/(a signal value of the second image signal).
3. The white balance adjustment device according to claim 1 ,
wherein the mixture ratio calculation portion calculates a flashed light ratio, as the mixture ratio, according to one of the following formulas:
(the flashed light ratio)= 1 −(a signal value of the first image signal)/(a signal value of the second image signal); and
(the flashed light ratio)=(the signal value of the second image signal−the signal value of the first image signal)/(the signal value of the second image signal).
4. The white balance adjustment device according to claim 1 ,
wherein the WB processing portion obtains, as the final image signal, an image signal having a signal value of Sout, which is continuously calculated by any one of:
Sout
=
S
2
+
w
1
·
(
S
1
-
S
2
)
;
Sout
=
w
1
·
S
1
+
(
1
-
w
1
)
·
S
2
;
Sout
=
S
1
+
w
2
·
(
S
2
-
S
1
)
;
Sout
=
(
1
-
w
2
)
·
S
1
+
w
2
·
S
2
;
and
Sout
=
w
1
·
S
1
+
w
2
·
S
2
,
wherein w 1 (0<w 1 <1) is an external light ratio, w 2 (0<w 2 <1) is a flashed light ratio, S 1 is the signal value of the third image signal, S 2 is the signal value of the fourth image signal, and Sout is the signal value of the final image signal.
5. The white balance adjustment device according to claim 1 , further comprising:
an LPF portion operable to perform low pass filter processing on the calculated mixture ratio to calculate a smoothed mixture ratio,
wherein the WB processing portion uses the smoothed mixture ratio as the mixture ratio.
6. A white balance adjustment device comprising:
a mixture ratio calculation portion operable to calculate a mixture ratio of an external light component and a flashed light component that are present in a second image signal, the mixture ratio being calculated based on a first image signal that is obtained without irradiating a flashed light on an object to be captured, and the second image signal that is obtained by illuminating the flashed light on the object to be captured;
an external light WB coefficient determination portion operable to determine an external light WB coefficient, which is a white balance coefficient for the external light, the external light WB coefficient being determined from the first image signal;
a flashed light WB coefficient setting portion operable to set a flashed light WB coefficient, which is a white balance coefficient of the flashed light; and
a WB processing portion operable to continuously perform white balance adjustment of the second image signal between the external light WB coefficient and the flashed light WB coefficient by using the calculated mixture ratio as an interpolation ratio,
wherein the mixture ratio calculation portion estimates an R component mixture ratio based on a first R image signal, which is an R component signal of the first image signal, and a second R image signal, which is an R component signal of the second image signal, estimates a G component mixture ratio based on a first G image signal, which is a G component signal of the first image signal, and a second G image signal, which is a G component signal of the second image signal, and estimates a B component mixture ratio based on a first B image signal, which is a B component signal of the first image signal, and a second B image signal, which is a B component signal of the second image signal,
wherein the external light WB coefficient determination portion determines an external light R component WB coefficient, an external light G component WB coefficient, and an external light B component WB coefficient from the first R image signal, the first G image signal, and the first B image signal, respectively,
wherein the flashed light WB coefficient setting portion sets a flashed light R component WB coefficient, a flashed light G component WB coefficient, and a flashed light B component WB coefficient, and
wherein the WB processing portion includes:
an external light WB correction portion operable to obtain a third R image signal, which is an R component signal of a third image signal, a third G image signal, which is a G component signal of the third image signal, and a third B image signal, which is a B component signal of the third image signal, by performing white balance correction on the second R image signal, the second G image signal, and the second B image signal based on the external light R component WB coefficient, the external light G component WB coefficient, and the external light B component WB coefficient, respectively;
a flashed light WB correction portion operable to obtain a fourth R image signal, which is an R component signal of a fourth image signal, a fourth G image signal, which is a G component signal of the fourth image signal, and a fourth B image signal, which is a B component signal of the fourth image signal, by performing white balance correction on the second R image signal, the second G image signal, and the second B image signal based on the flashed light R component WB coefficient, the flashed light G component WB coefficient, and the flashed light B component WB coefficient, respectively; and
an interpolation portion operable to obtain a final R image signal, which takes on a continuous value, by interpolating a signal value of the third R image signal and a signal value of the fourth R image signal, based on the R component mixture ratio, obtain a final G image signal, which takes on a continuous value, by interpolating a signal value of the third G image signal and a signal value of the fourth G image signal, based on the G component mixture ratio, and obtain a final B image signal, which takes on a continuous value, by interpolating a signal value of the third B image signal and a signal value of the fourth B image signal, based on the B component mixture ratio.
7. An image capture device comprising the white balance adjustment device according to claim 1 .
8. The image capture device according to claim 7 , further comprising a flash emission portion operable to irradiate the flashed light onto the object to be captured.
9. A white balance adjustment method, comprising:
calculating a mixture ratio of an external light component and a flashed light component that are present in a second image signal, the mixture ratio being calculated based on a first image signal that is obtained without irradiating a flashed light on an object to be captured, and the second image signal that is obtained by illuminating the flashed light on the object to be captured;
determining an external light WB coefficient, which is a white balance coefficient for the external light, the external light WB coefficient being determined from the first image signal;
setting a flashed light WB coefficient, which is a white balance coefficient of the flashed light; and
using the calculated mixture ratio as an interpolation ratio to continuously perform white balance adjustment of the second image signal between the external light WB coefficient and the flashed light WB coefficient,
wherein the using of the calculated mixture ratio to continuously perform the white balance adjustment includes:
obtaining a third image signal by performing white balance correction on the second image signal based on the external light WB coefficient;
obtaining a fourth image signal by performing white balance correction on the second image signal based on the flashed light WB coefficient; and
obtaining a final image signal, which takes on a continuous value, by interpolating a signal value of the third image signal and a signal value of the fourth image signal, based on the calculated mixture ratio.
10. A non-transitory computer-readable storage medium that stores a program for causing a computer to function as:
a mixture ratio calculation portion operable to calculate a mixture ratio of an external light component and a flashed light component that are present in a second image signal, the mixture ratio being calculated based on a first image signal that is obtained without irradiating a flashed light on an object to be captured, and the second image signal that is obtained by illuminating the flashed light on the object to be captured;
an external light WB coefficient determination portion operable to determine an external light WB coefficient, which is a white balance coefficient for the external light, the external light WB coefficient being determined from the first image signal;
a flashed light WB coefficient setting portion operable to set a flashed light WB coefficient, which is a white balance coefficient of the flashed light; and
a WB processing portion operable to continuously perform white balance adjustment of the second image signal between the external light WB coefficient and the flashed light WB coefficient by using the calculated mixture ratio as an interpolation ratio,
wherein the WB processing portion obtains a third image signal by performing white balance correction on the second image signal based on the external light WB coefficient,
wherein the WB processing portion obtains a fourth image signal by performing white balance correction on the second image signal based on the flashed light WB coefficient, and
wherein the WB processing portion obtains a final image signal, which takes on a continuous value, by interpolating a signal value of the third image signal and a signal value of the fourth image signal based on the calculated mixture ratio.
11. An integrated circuit, comprising:
a mixture ratio calculation portion operable to calculate a mixture ratio of an external light component and a flashed light component that are present in a second image signal, the mixture ratio being calculated based on a first image signal that is obtained without irradiating a flashed light on an object to be captured, and the second image signal that is obtained by illuminating the flashed light on the object to be captured;
an external light WB coefficient determination portion operable to determine an external light WB coefficient, which is a white balance coefficient for the external light, the external light WB coefficient being determined from the first image signal;
a flashed light WB coefficient setting portion operable to set a flashed light WB coefficient, which is a white balance coefficient of the flashed light; and
a WB processing portion operable to continuously perform white balance adjustment of the second image signal between the external light WB coefficient and the flashed light WB coefficient by using the calculated mixture ratio as an interpolation ratio,
wherein the WB processing portion obtains a third image signal by performing white balance correction on the second image signal based on the external light WB coefficient,
wherein the WB processing portion obtains a fourth image signal by performing white balance correction on the second image signal based on the flashed light WB coefficient, and
wherein the WB processing portion obtains a final image signal, which takes on a continuous value, by interpolating a signal value of the third image signal and a signal value of the fourth image signal based on the calculated mixture ratio.