Image interpolation method and apparatus using pattern characteristics of color filter array
View Patent ↗An image interpolating method and apparatus, in which horizontal and vertical Differences of Absolute Inter-channel Differences (DAIDs) are calculated from a color filter array (CFA) image, and an unknown pixel is interpolated in horizontal and vertical directions estimated in consideration of the DAIDs of R, G, and B pixels. Therefore, the image interpolating method and apparatus provide a large Peak Signal and Noise Ratio (PSNR), a Structural Similarity (SSIM), and high visual quality images.
1. An image interpolating method comprising:
detecting an edge direction by calculating absolute inter-channel differences with respect to left, right, up, and down directions of a central pixel of a color filter array according to pattern characteristics of R, G, and B pixels arranged in the color filter array, the absolute inter-channel differences being calculated by subtracting the absolute value of a difference between a color intensity value of a pixel of a first color channel on one side of the central pixel and a color intensity value of a neighboring pixel of a second color channel on the one side of the central pixel, from the absolute value of a difference between a color intensity value of a pixel of the first color channel on a symmetrically opposite side of the central pixel and a color intensity value of a neighboring pixel of the second color channel on the symmetrically opposite side of the central pixel;
calculating directional interpolation weights for the left, right, up, and down directions, respectively, according to the pattern characteristics of the R, G, and B pixels; and
interpolating an unknown pixel value using color differences between the R, G, and B pixels with respect to the left, right, up, and down directions of the center pixel and the directional interpolation weights according to the detected edge direction.
2. The image interpolating method of claim 1 , wherein the absolute inter-channel differences are differences between values of pixels that are adjacent to each other in the left, right, up, and down directions.
3. The image interpolating method of claim 2 , wherein the detecting of the edge direction comprises calculating a horizontal difference D H for the left and right directions and a vertical difference D V for the up and down directions from the absolute inter-channel differences with respect to the left, right, up, and down directions.
4. The image interpolating method of claim 3 , wherein the detecting of the edge direction comprises, if the central pixel of the color filter array is R i,j , calculating the horizontal difference D H and the vertical difference D V according to the following Equations 4 and 5:
D H =D H L −D H R =|R i,j−2 −G i,j−1 |−|R i,j+2 |−|R i,j+2 −G i,j+1 | [Equation 4]
D V =D V U −D V D =|R i−2,j −G i−1,j |−|R i+2,j −G i+1,j | [Equation 5]
where R p.q and G p,q denote color intensity values at positions (p,q) in the R and G channels, respectively, and D H L , D H R , D V U , and D V D denote inter-channel differences with respect to the left, right, up, and down directions, respectively.
5. The image interpolating method of claim 4 , wherein the interpolating of the unknown pixel value comprises, if the central pixel of the color filter array is R i,j , interpolating an unknown G pixel value according to the following Equation 6:
G
i
,
j
′
=
+
{
R
i
,
j
+
γ
H
L
Δ
G
H
L
+
γ
H
R
Δ
G
H
R
γ
H
L
+
γ
H
R
,
D
H
<
D
V
R
i
,
j
+
γ
V
U
Δ
G
V
U
+
γ
V
D
Δ
G
V
D
γ
V
U
+
γ
V
D
,
D
H
>
D
V
R
i
,
j
+
γ
H
L
Δ
G
H
L
+
γ
H
R
Δ
G
H
R
+
γ
V
U
Δ
G
V
U
+
γ
V
D
Δ
G
V
D
γ
H
L
+
γ
H
R
+
γ
V
U
+
γ
V
D
,
otherwise
Equation
6
where G′ i,j denotes an interpolated G pixel value, ΔG H L , ΔG H R , ΔG V U , and ΔG V D denote color differences defined with respect to the left, right, up, and down directions, respectively, and γ H L , γ H R , γ V U , and γ V D denote directional interpolation weights defined with respect to the left, right, up, and down directions, respectively.
6. The image interpolating method of claim 3 , wherein the interpolating of the unknown pixel value comprises, after a G pixel from among unknown pixels is interpolated, interpolating the values of the other unknown pixels using a pixel value G′ of the interpolated G pixel.
7. The image interpolating method of claim 6 , wherein the interpolating of the unknown pixel value comprises, if the central pixel of the color filter array is B i,j , interpolating an unknown R pixel value according to the following Equation 15:
R
i
,
j
′
=
G
i
,
j
′
+
γ
H
L
Δ
R
H
L
+
γ
H
R
Δ
R
H
R
+
γ
V
U
Δ
R
V
U
+
γ
V
D
Δ
R
V
D
γ
H
L
+
γ
H
R
+
γ
V
U
+
γ
V
D
[
Equation
15
]
where R′ i,j and G′ i,j denotes interpolated pixel values, ΔR H L , ΔR H R , ΔR V U , and ΔR V D denote color differences defined with respect to the left, right, up, and down directions of the center pixel, respectively, ΔR H L =R i+1,j−1 −G′ i+1,j−1 , ΔR H R =R i−1,j+1 −G′ i−1,j+1 , ΔR V U =R i−1,j−1 −G′ i−1,j−1 , and ΔR V D =R i+1,j+1 −G′ i+1,j+1 , and γ H L , γ H R , γ V U , and γ V D denote directional interpolation weights defined with respect to the left, right, up, and down directions, respectively.
8. The image interpolating method of claim 7 , wherein, if the central pixel of the color filter array is G i,j , the color differences defined with respect to the left, right, up, and down directions in Equation 15 are ΔR H L =R′ i,j−1 −G′ i,j−1 , ΔR H R =R′ i,j+1 −G′ i,j+1 , ΔR V U =R i+1,j −G′ i−1,j , and ΔR V D =R i+1,j −G′ i+1,j , respectively.
9. The image interpolating method of claim 1 , further comprising adjusting the interpolated pixel value by adding a high frequency component of a known pixel to the interpolated pixel value.
10. A non-transitory computer readable recording medium encoded with computer-executable instructions that when executed by a computer perform the following method:
detecting an edge direction by calculating absolute inter-channel differences with respect to left, right, up, and down directions of a central pixel of a color filter array according to pattern characteristics of R, G, and B pixels arranged in the color filter array, the absolute inter-channel differences being calculated by subtracting the absolute value of a difference between a color intensity value of a pixel of a first color channel on one side of the central pixel and a color intensity value of a neighboring pixel of a second color channel on the one side of the central pixel, from the absolute value of a difference between a color intensity value of a pixel of the first color channel on a symmetrically opposite side of the central pixel and a color intensity value of a neighboring pixel of the second color channel on the symmetrically opposite side of the central pixel;
calculating directional interpolation weights for the left, right, up, and down directions, respectively, according to the pattern characteristics of the R, G, and B pixels; and
interpolating an unknown pixel value using color differences between the R, G, and B pixels with respect to the left, right, up, and down directions of the center pixel and the directional interpolation weights according to the detected edge direction.
11. An image interpolating apparatus comprising:
an edge direction detection unit configured to detect an edge direction by calculating absolute inter-channel differences with respect to left, right, up, and down directions of a central pixel of a color filter array according to pattern characteristics of R, G, and B pixels arranged in the color filter array, the absolute inter-channel differences being calculated by subtracting the absolute value of a difference between a color intensity value of a pixel of a first color channel on one side of the central pixel and a color intensity value of a neighboring pixel of a second color channel on the one side of the central pixel, from the absolute value of a difference between a color intensity value of a pixel of the first color channel on a symmetrically opposite side of the central pixel and a color intensity value of a neighboring pixel of the second color channel on the symmetrically opposite side of the central pixel;
a weight calculation unit configured to calculate directional interpolation weights for the left, right, up, and down directions, respectively, according to the pattern characteristics of the R, G, and B pixels; and
an interpolation unit configured to interpolate an unknown pixel value using color differences between the R, G, and B pixels with respect to the left, right, up, and down directions and the directional interpolation weights according to the detected edge direction.
12. The image interpolating apparatus of claim 11 , wherein the absolute inter-channel differences are differences between values of pixels that are adjacent to each other in the left, right, up, and down directions.
13. The image interpolating apparatus of claim 12 , wherein the edge direction detection unit calculates a horizontal difference D H for the left and right directions and a vertical difference D V for the up and down directions from the absolute inter-channel differences with respect to the left, right, up, and down directions.
14. The image interpolating apparatus of claim 13 , wherein, if the central pixel of the color filter array is R i,j the edge direction detection unit calculates the horizontal difference D H and the vertical difference D V according to the following Equations 4 and 5:
D H =D H L −D H R −|R i,j−2 −G i,j−1 |−|R i,j+2 −G i,j+1 | [Equation 4]
D V =D V U −D V D =|R i−2,j −G i−1,j |−|R i+2,j −G i+1,j | [Equation 5]
where R p.q and G p,q denote color intensity values at positions (p,q) in the R and G channels, respectively, and ΔD H L , ΔD H R , ΔD V U , and D V D denote inter-channel differences with respect to the left, right, up, and down directions of the center pixel, respectively.
15. The image interpolating apparatus of claim 14 , wherein, if the central pixel of the color filter array is R i,j , the interpolation unit interpolates an unknown G pixel value according to the following Equation 6:
G
i
,
j
′
=
+
{
R
i
,
j
+
γ
H
L
Δ
G
H
L
+
γ
H
R
Δ
G
H
R
γ
H
L
+
γ
H
R
,
D
H
<
D
V
R
i
,
j
+
γ
V
U
Δ
G
V
U
+
γ
V
D
Δ
G
V
D
γ
V
U
+
γ
V
D
,
D
H
>
D
V
R
i
,
j
+
γ
H
L
Δ
G
H
L
+
γ
H
R
Δ
G
H
R
+
γ
V
U
Δ
G
V
U
+
γ
V
D
Δ
G
V
D
γ
H
L
+
γ
H
R
+
γ
V
U
+
γ
V
D
,
otherwise
Equation
6
where G′ i,j denotes an interpolated G pixel value, ΔG H L , ΔG H R , ΔG V U , and ΔG V D denote color differences defined with respect to the left, right, up, and down directions, respectively, and γ H L , γ H R , γ V U , and γ V D denote directional interpolation weights defined with respect to the left, right, up, and down directions, respectively.
16. The image interpolating apparatus of claim 13 , wherein, after a G pixel from among unknown pixels is interpolated, the interpolation unit interpolates the values of the other unknown pixels using a pixel value G′ of the interpolated G pixel.
17. The image interpolating apparatus of claim 16 , wherein, if the central pixel of the color filter array is B i,j , the interpolation unit interpolates an unknown R pixel value according to the following Equation 15:
R
i
,
j
′
=
G
i
,
j
′
+
γ
H
L
Δ
R
H
L
+
γ
H
R
Δ
R
H
R
+
γ
V
U
Δ
R
V
U
+
γ
V
D
Δ
R
V
D
γ
H
L
+
γ
H
R
+
γ
V
U
+
γ
V
D
[
Equation
15
]
where R′ i,j and G′ i,j denotes interpolated pixel values, ΔR H L , ΔR H R , ΔR V U , and ΔR V D denote color differences defined with respect to the left, right, up, and down directions of the center pixel, respectively, ΔR H L =R i+1,j−1 −G′ i+1,j−1 , ΔR H R =R i−1,j+1 −G′ i−1,j+1 , ΔR V U =R i−1,j−1 −G′ i−1,j−1, and γ H L , γ H R , γ V U , and γ V D denote directional interpolation weights defined with respect to the left, right, up, and down directions, respectively.
18. The image interpolating apparatus of claim 17 , wherein, if the central pixel of the color filter array is G i,j , the color differences defined with respect to the left, right, up, and down directions in Equation 15 are ΔR H L =R′ i,j−1 −G′ i,j−1 , ΔR H R =R′ i,j+1 −G′ i,j+1 , ΔR V U −R i−1,j −G′ i−1,j , and ΔR V D =R i+1,j −G′ i+1,j , respectively.
19. The image interpolating apparatus of claim 11 , further comprising an adjusting unit for adjusting the interpolated pixel value by adding a high frequency component of a known pixel to the interpolated pixel value.
20. An image interpolating method comprising:
detecting an edge direction by calculating absolute inter-channel differences with respect to left, right, up, and down directions of a central pixel of a color filter array according to pattern characteristics of different color pixels arranged in the color filter array, the absolute inter-channel differences being calculated by subtracting the absolute value of a difference between a color intensity value of a pixel of a first color channel on one side of the central pixel and a color intensity value of a neighboring pixel of a second color channel on the one side of the central pixel, from the absolute value of a difference between a color intensity value of a pixel of the first color channel on a symmetrically opposite side of the central pixel and a color intensity value of a neighboring pixel of the second color channel on the symmetrically opposite side of the central pixel;
calculating directional interpolation weights for the left, right, up, and down directions, respectively, according to the pattern characteristics of the different color pixels; and
interpolating an unknown pixel value using color differences between the different color pixels with respect to the left, right, up, and down directions of the center pixel and the directional interpolation weights according to the detected edge direction.