Spatio-temporal noise removal method using block classification and display device using the same
View Patent ↗A spatio-temporal noise removal method using block classification that removes noise from a third field by using first, second, and third fields which are continuously inputted. The method includes generating first, second, and third motion-compensated fields; classifying blocks of the third field into a uniform region and a non-uniform region based on a variance value of a generated difference image; performing temporal filtering over every block of the third field; performing spatio-temporal filtering over every block of the first and third motion-compensated fields and the third field and performing the temporal filtering over the every block of the third field based on spatially-filtered value; and outputting a third noise-removed field by applying a weighted value to the temporal-filtered value and the spatio-temporal-filtered value.
1. A display device for displaying images by removing noise from a third field by using a first noise-removed field, a second noise-removed field, and the third field, comprising:
a motion calculation part adapted to generate a first motion-compensated field by using the third field and the second noise-removed field, generate a second motion-compensated field by using the first and second noise-removed fields, and generate a third motion-compensated field by using the second motion-compensated field and the third field;
a classification part adapted to generate blocks of the third field into a uniform region and a non-uniform region based on a variance value of a generated difference image by using the third field and any of the first and second motion-compensated fields;
a temporal filter adapted to perform temporal filtering over every block of the third field based on the first and third motion-compensated fields, the third field, and the variance value;
a spatio-temporal filter adapted to perform spatio-temporal filtering by performing spatial filtering over every block of the first and third motion-compensated fields and the third field, and by performing the temporal filtering over every block of the third field based on the spatially-filtered value; and
an arithmetic logic unit adapted to output a third noise-removed field by applying a weighted value to the temporal-filtered value and the spatio-temporal-filtered value, respectively, depending on whether a block in the third field is the uniform region or the non-uniform region.
2. The display device as claimed in claim 1 , wherein the motion calculation part comprises:
a first motion calculation unit adapted to generate the first motion-compensated field by using the third field and the second noise-removed field;
a second motion calculation unit adapted to generate the second motion-compensated field by using the first and second noise-removed fields; and
a third motion calculation unit adapted to generate the third motion-compensated field by using the second motion-compensated field and the third field.
3. The display device as claimed in claim 1 , wherein the classification part includes:
an adder adapted to generate a difference image by calculating a pixel value difference between the third field and any of the first and second motion-compensated fields;
a noise estimation unit adapted to estimate, as noise, a variance value having a maximum frequency after calculating a variance value over every block of the difference image; and
a region classification unit adapted to classify, as the uniform region, a block having a variance value higher than the estimated noise.
4. The display device as claimed in claim 1 , further comprising a weight value unit adapted to generate a weight value in order for an output value of the spatio-temporal filter to be taken into more account than an output value of the temporal filter if an arbitrary block of the third field is the uniform region, and generate a weight value in order for the output value of the temporal filter to be taken into more account than the output value of the spatio-temporal filter.
5. The display device as claimed in claim 1 , wherein the arithmetic logic unit outputs the third noise-removed field by using an equation as below:
{circumflex over (f)} ( x,y,k )= w×f t ( x,y,k )+(1 −w )× f s−t ( x,y,k ),
here, {circumflex over (f)}(x,y,k) denotes the third noise-removed field, w and (1−w) denote a weight value applied to a temporal-filtered value and a weight value applied to a spatio-temporal-filtered value, respectively, f t (x,y,k) denotes an output value of the temporal filter, and f s−t (x,y,k) denotes an output value of the spatio-temporal filter.
6. The display device as claimed in claim 1 , wherein the spatio-temporal filter performs the spatio-temporal filtering over every block of the third field by using an equation as below:
f
^
s
-
t
(
x
,
y
,
k
)
=
1
3
{
f
^
1
s
(
x
,
y
,
k
)
+
f
^
2
s
(
x
,
y
,
k
)
+
f
^
s
(
x
,
y
,
k
)
}
,
here, {circumflex over (f)} s−t (x,y,k) denotes an output value of the spatio-temporal filter,
f
^
1
s
(
x
,
y
,
k
)
,
f
^
2
s
(
x
,
y
,
k
)
,
and
f
^
s
(
x
,
y
,
k
)
denote spatially-filtered values over the first motion-compensated field, the third motion-compensated field, and the third field, respectively, and (x,y,k) denotes a pixel position.
7. The display device as claimed in claim 1 , wherein the temporal filter performs the temporal filtering over every block of the third field by using an equation as below:
f
^
t
(
x
,
y
,
k
)
=
σ
f
2
(
x
,
y
,
k
)
σ
f
2
(
x
,
y
,
k
)
+
σ
n
2
(
x
,
y
,
k
)
{
g
(
x
,
y
,
k
)
-
E
[
g
(
x
,
y
,
k
)
]
}
+
E
[
g
(
x
,
y
,
k
)
]
,
here
,
E
[
g
(
x
,
y
,
k
)
]
=
1
3
(
g
(
x
,
y
,
k
)
+
f
^
1
mc
(
x
,
y
,
k
)
+
f
^
2
mc
(
x
,
y
,
k
)
)
,
f
^
t
(
x
,
y
,
k
)
denotes the temporal-filtered value, g(x,y,k) the third field,
f
^
1
mc
(
x
,
y
,
k
)
the first motion-compensated field,
f
^
2
mc
(
x
,
y
,
k
)
the third motion-compensated field, (x,y,k) a pixel position in an arbitrary block, σ n 2 , a variance value of the uniform region, and σ f 2 a variance value of a filtered block.
8. The display device as claimed in claim 1 , wherein the spatio-temporal filter and the temporal filter are a Linear Minimum Mean Square Error (LMMSE) filter.
9. A spatio-temporal noise removal method using block classification for noise removal from a third field by using a first noise-removed field, a second noise-removed field, and the third field, comprising:
generating a first motion-compensated field by using the third field and the second noise-removed field, generating a second motion-compensated field by using the first and second noise-removed fields, and generating a third motion-compensated field by using the second motion-compensated field and the third field;
classifying blocks of the third field into a uniform region and a non-uniform region based on a variance value of a generated difference image by using the third field and any of the first and second motion-compensated fields;
performing temporal filtering over every block of the third field based on the first and third motion-compensated fields, the third field, and the variance value;
performing spatio-temporal filtering by performing spatial filtering over every block of the first and third motion-compensated fields and the third field and performing the temporal filtering over the every block of the third field based on the spatially-filtered value; and
outputting a third noise-removed field by applying a weighted value to the temporal-filtered value and the spatio-temporal-filtered value, respectively, depending on whether a block in the third field is the uniform region or the non-uniform region.
10. The spatio-temporal noise removal method as claimed in claim 9 , wherein generating the motion-compensated field comprises:
generating the first motion-compensated field by using the third field and the second noise-removed field;
generating the second motion-compensated field by using the first and second noise-removed fields; and
generating the third motion-compensated field being a motion-compensated field by using the second motion-compensated field and the third field.
11. The spatio-temporal noise removal method as claimed in claim 9 , wherein classifying blocks of the third field comprises:
generating a difference image by calculating a pixel value difference between the third field and any of the first and second motion-compensated fields;
estimating as noise a variance value having a maximum frequency after calculating a variance value over every block of the difference image; and
classifying as the uniform region a block having a variance value larger than the estimated noise.
12. The spatio-temporal noise removal method as claimed in claim 9 , further comprising:
generating a weight value in order for an output value of the spatio-temporal filter to be taken into more account than an output value of the temporal filter if an arbitrary block of the third field is the uniform region, and
generating a weight value in order for the output value of the temporal filter to be taken into more account than the output value of the spatio-temporal filter.
13. The spatio-temporal noise removal method as claimed in claim 9 , wherein the third noise-removed field is calculated by an equation as below:
{circumflex over (f)} ( x,y,k )= w×f t ( x,y,k )+(1 −w )× f s−t ( x,y,k ),
in here, {circumflex over (f)}(x,y,k) denotes the third noise-removed field, w and (1−w) denote a weight value applied to a temporal-filtered value and a weight value applied to a spatio-temporal-filtered value, respectively, f t (x,y,k) denotes a temporal-filtered value, and f s−t (x,y,k) denotes a spatio-temporal-filtered value.
14. The spatio-temporal noise method as claimed in claim 9 , wherein in performing the spatio-temporal filtering over every block of the third field, an equation, as below, is used:
{circumflex over (f)} s-t ( x,y,k )=1/3 { {circumflex over (f)} s 1 ( x,y,k )+ {circumflex over (f)} s 2 ( x,y,k )+ {circumflex over (f)} s ( x,y,k )}
here, {circumflex over (f)} s−t (x,y,k) denotes a spatio-temporal-filtered value,
f
^
1
s
(
x
,
y
,
k
)
,
f
^
2
s
(
x
,
y
,
k
)
,
and
f
^
s
(
x
,
y
,
k
)
denote spatially-filtered values over the first motion-compensated field, the third motion-compensated field, and the third field, respectively, and (x,y,k) denotes a pixel position.
15. The spatio-temporal noise removal method as claimed in claim 9 , wherein in performing the temporal filtering over every block of the third field, an equation, as below, is used:
f
^
t
(
x
,
y
,
k
)
=
σ
f
2
(
x
,
y
,
k
)
σ
f
2
(
x
,
y
,
k
)
+
σ
n
2
(
x
,
y
,
k
)
{
g
(
x
,
y
,
k
)
-
E
[
g
(
x
,
y
,
k
)
]
}
+
E
[
g
(
x
,
y
,
k
)
]
,
here
,
E
[
g
(
x
,
y
,
k
)
]
=
1
3
(
g
(
x
,
y
,
k
)
+
f
^
1
mc
(
x
,
y
,
k
)
+
f
^
2
mc
(
x
,
y
,
k
)
)
,
f
^
t
(
x
,
y
,
k
)
denotes the temporal-filtered value, g(x,y,k) the third field,
f
^
1
mc
(
x
,
y
,
k
)
the first motion-compensated field,
f
^
2
mc
(
x
,
y
,
k
)
the third motion-compensated field, (x,y,k) a pixel position in an arbitrary block, σ n 2 a variance value of the uniform region, and σ f 2 a variance value of a filtered block.