Method and apparatus providing hardware-efficient demosaicing of image data
View Patent ↗A method and apparatus for determining missing color pixel values in image data (i.e., for demosaicing image data) using a small number of lines in a buffer memory. Pixel values are interpolated in a horizontal or a vertical direction based on gradient scores. Interpolation direction selection can be refined based on a chroma scores or morphological operations. A chroma blurring technique to reduce false color artifacts is also disclosed. A disclosed embodiment can be implemented in an imager having as few as three buffer lines.
1. A method of determining a missing color value associated with a subject pixel in digital image data derived from a Bayer-patterned pixel array, the method comprising:
computing a horizontal gradient score, a horizontal chroma score, a vertical gradient score, and a vertical chroma score associated with the subject pixel;
adding the horizontal gradient score to the horizontal chroma score to yield a combined horizontal score, and adding the vertical gradient score to the vertical chroma score to yield a combined vertical score;
comparing the combined horizontal score to the combined vertical score to determine an interpolation direction; and
determining the missing color value by interpolating in the interpolation direction, wherein the subject pixel is a native red pixel, the horizontal gradient score, D h (r+1,c+1) , is computed as follows:
D h (r+1,c+1) =|B (r,c) −B (r,c+2) |+|B (r+2,c) −B (r+2,c+2) |+|G (r+1,c) −G (r+1,cc+2)
and the vertical gradient score, D v (r+1,c+1) , is computed as follows:
D v (r+1,c+1) =|B (r,c) −B (r+2,c) |+|B (r,c+2) −B (r+2,c+2) |+|G (r,c+1) −G (r+2,c+1) |
wherein B represents the blue value associated with the pixel located at the row and column indicated in the subscript, and G represents the green value associated with the pixel located at the row and column indicated in the subscript.
2. The method of claim 1 , wherein the missing color value is a green value, G (r+1,c+1) , computed as follows:
G
(
r
+
1
,
c
+
1
)
=
{
G
(
r
+
1
,
c
)
+
G
(
r
+
1
,
c
+
2
)
2
if
D
h
(
r
+
1
,
c
+
1
)
′
<
D
v
(
r
+
1
,
c
+
1
)
′
G
(
r
,
c
+
1
)
+
G
(
r
+
2
,
c
+
1
)
2
if
D
h
(
r
+
1
,
c
+
1
)
′
>
D
v
(
r
+
1
,
c
+
1
)
′
G
(
r
+
1
,
c
)
+
G
(
r
+
1
,
c
+
2
)
+
G
(
r
,
c
+
1
)
+
G
(
r
+
2
,
c
+
1
)
4
if
D
h
(
r
+
1
,
c
+
1
)
′
=
D
v
(
r
+
1
,
c
+
1
)
′
wherein G represents the green value associated with the pixel located at the row and column indicated in the subscript, D′ h (r+1,c+1) represents the combined horizontal score and D′ v (r+1,c+1) represents the combined vertical score.
3. The method of claim 1 , wherein the horizontal combined score equals the vertical combined score, and the missing color value is a blue value, B (r+1,c+1) , computed as follows:
B
(
r
+
1
,
c
+
1
)
=
(
B
(
r
,
c
)
+
B
(
r
,
c
+
2
)
+
B
(
r
+
2
,
c
)
+
B
(
r
+
2
,
c
+
2
)
)
4
wherein B represents the blue value associated with the pixel located at the row and column indicated in the subscript.
4. A method of determining a missing color value associated with a subject pixel in digital image data derived from a Bayer-patterned pixel array, the method comprising:
computing a horizontal gradient score, a horizontal chroma score, a vertical gradient score, and a vertical chroma score associated with the subject pixel;
adding the horizontal gradient score to the horizontal chroma score to yield a combined horizontal score, and adding the vertical gradient score to the vertical chroma score to yield a combined vertical score;
comparing the combined horizontal score to the combined vertical score to determine an interpolation direction; and
determining the missing color value by interpolating in the interpolation direction, wherein subject pixel is a native red pixel, the horizontal chroma score, K h (r+1,c+1) , is computed as follows:
K
h
(
r
+
1
,
c
+
1
)
=
G
(
r
+
1
,
c
+
2
)
+
G
(
r
+
1
,
c
)
2
-
R
(
r
+
1
,
c
+
1
)
and the vertical chroma score, K v (r+1,c+1) , is computed as follows:
K
v
(
r
+
1
,
c
+
1
)
=
G
(
r
+
2
,
c
+
1
)
+
G
(
r
,
c
+
1
)
2
-
R
(
r
+
1
,
c
+
1
)
wherein G represents the green value associated with the pixel located at the row and column indicated in the subscript, and R represents the red value associated with the pixel located at the row and column indicated in the subscript.
5. A method of determining a missing color value associated with a subject pixel in digital image data derived from a Bayer-patterned pixel array, the method comprising:
computing a horizontal gradient score, a horizontal chroma score, a vertical gradient score, and a vertical chroma score associated with the subject pixel;
adding the horizontal gradient score to the horizontal chroma score to yield a combined horizontal score, and adding the vertical gradient score to the vertical chroma score to yield a combined vertical score;
comparing the combined horizontal score to the combined vertical score to determine an interpolation direction; and
determining the missing color value by interpolating in the interpolation direction, wherein the subject pixel is a native red pixel, the interpolation direction is horizontal, and the missing color value is a blue value, B (r+1,c+1) , computed as follows:
B
(
r
+
1
,
c
+
1
)
=
G
(
r
+
1
,
c
+
1
)
+
B
(
r
,
c
+
1
)
′
-
G
(
r
,
c
+
1
)
+
B
(
r
+
2
,
c
+
1
)
′
-
G
(
r
+
2
,
c
+
1
)
2
B
(
r
,
c
+
1
)
′
=
B
(
r
,
c
)
+
B
(
r
,
c
+
2
)
2
B
(
r
+
2
,
c
+
1
)
′
=
B
(
r
+
2
,
c
)
+
B
(
r
+
2
,
c
+
2
)
2
wherein G represents the green value associated with the pixel located at the row and column indicated in the subscript, and B represents the blue value associated with the pixel located at the row and column indicated in the subscript.
6. A method of determining a missing color value associated with a subject pixel in digital image data derived from a Bayer-patterned pixel array, the method comprising:
computing a horizontal gradient score, a horizontal chroma score, a vertical gradient score, and a vertical chroma score associated with the subject pixel;
adding the horizontal gradient score to the horizontal chroma score to yield a combined horizontal score, and adding the vertical gradient score to the vertical chroma score to yield a combined vertical score;
comparing the combined horizontal score to the combined vertical score to determine an interpolation direction; and
determining the missing color value by interpolating in the interpolation direction, wherein the subject pixel is a native red pixel, the interpolation direction is vertical, and the missing color value is a blue value, B (r+1,c+1) , computed as follows:
B
(
r
+
1
,
c
+
1
)
=
G
(
r
+
1
,
c
+
1
)
+
B
(
r
+
1
,
c
)
′
-
G
(
r
+
1
,
c
)
+
B
(
r
+
1
,
c
+
2
)
′
-
G
(
r
+
1
,
c
+
2
)
2
B
(
r
+
1
,
c
)
′
=
B
(
r
,
c
)
+
B
(
r
+
2
,
c
)
2
B
(
r
+
1
,
c
+
2
)
′
=
B
(
r
,
c
+
2
)
+
B
(
r
+
2
,
c
+
2
)
2
wherein G represents the green value associated with the pixel located at the row and column indicated in the subscript, and B represents the blue value associated with the pixel located at the row and column indicated in the subscript.
7. A method of determining a missing color value associated with a subject pixel in digital image data derived from a Bayer-patterned pixel array, the method comprising:
computing a horizontal gradient score, a horizontal chroma score, a vertical gradient score, and a vertical chroma score associated with the subject pixel;
adding the horizontal gradient score to the horizontal chroma score to yield a combined horizontal score, and adding the vertical gradient score to the vertical chroma score to yield a combined vertical score;
comparing the combined horizontal score to the combined vertical score to determine an interpolation direction; and
determining the missing color value by interpolating in the interpolation direction, wherein the subject pixel is a native green pixel in a row comprising native red and green pixels, green values for immediately adjacent pixels in the same row as the subject pixel were interpolated horizontally, and the missing color value is a blue value, B (r+1,c) , computed as follows:
G
(
r
,
c
)
′
=
G
(
r
,
c
-
1
)
+
G
(
r
,
c
+
1
)
2
G
(
r
+
2
,
c
)
′
=
G
(
r
+
2
,
c
-
1
)
+
G
(
r
+
2
,
c
+
1
)
2
B
(
r
+
1
,
c
)
=
G
(
r
+
1
,
c
)
+
B
(
r
,
c
)
-
G
(
r
,
c
)
′
+
B
(
r
+
2
,
c
)
-
G
(
r
+
2
,
c
)
′
2
wherein G represents the green value associated with the pixel located at the row and column indicated in the subscript, and B represents the blue value associated with the pixel located at the row and column indicated in the subscript.
8. A method of determining a missing color value associated with a subject pixel in digital image data derived from a Bayer-patterned pixel array, the method comprising:
computing a horizontal gradient score, a horizontal chroma score, a vertical gradient score, and a vertical chroma score associated with the subject pixel;
adding the horizontal gradient score to the horizontal chroma score to yield a combined horizontal score, and adding the vertical gradient score to the vertical chroma score to yield a combined vertical score;
comparing the combined horizontal score to the combined vertical score to determine an interpolation direction;
determining the missing color value by interpolating in the interpolation direction;
replacing the color values associated with the subject pixel, R (r,c) , G (r,c) , B (r,c) , with adjusted color values R′ (r,c) , G′ (r,c) , B′ (r,c) , computed as follows:
R
_
(
r
,
c
)
=
R
(
r
,
c
-
1
)
+
R
(
r
,
c
)
+
R
(
r
,
c
+
1
)
3
G
_
(
r
,
c
)
=
G
(
r
,
c
-
1
)
+
G
(
r
,
c
)
+
G
(
r
,
c
+
1
)
3
B
_
(
r
,
c
)
=
B
(
r
,
c
-
1
)
+
B
(
r
,
c
)
+
B
(
r
,
c
+
1
)
3
Y
(
r
,
c
)
=
R
(
r
,
c
)
4
+
G
(
r
,
c
)
2
+
B
(
r
,
c
)
4
C
b
(
r
,
c
)
=
B
_
(
r
,
c
)
-
G
_
(
r
,
c
)
C
r
(
r
,
c
)
=
R
_
(
r
,
c
)
-
G
_
(
r
,
c
)
R
(
r
,
c
)
′
=
Y
(
r
,
c
)
+
3
4
C
r
(
r
,
c
)
-
1
4
C
b
(
r
,
c
)
G
(
r
,
c
)
′
=
Y
(
r
,
c
)
-
1
4
C
r
(
r
,
c
)
-
1
4
C
b
(
r
,
c
)
B
(
r
,
c
)
′
=
Y
(
r
,
c
)
-
1
4
C
r
(
r
,
c
)
+
3
4
C
b
(
r
,
c
)
wherein R represents the red value associated with the pixel located at the row and column indicated in the subscript, B represents the blue value associated with the pixel located at the row and column indicated in the subscript, and G represents the green value associated with the pixel located at the row and column indicated in the subscript.