Video denoising system based on noise correlation
The present invention discloses a video denoising system based on noise correlation, said system comprises a unit for estimating correlation of noises of adjacent pixels, which receives input of the inter-frame difference and motion probability, and estimates correlation of noises of adjacent pixels according to correlation of inter-frame differences between adjacent pixels in a still region, and outputs a noise correlation coefficient; a maximum filtering weight adjusting unit, which adaptively adjusts the maximum weight for temporal filtering according to the noise correlation coefficient and outputs the maximum weight for temporal filtering; the maximum weight for temporal filtering can control the range of fluctuation of the temporal filtering weight and the difference between the denoising effects for different pixels. The system of the present invention can solve the problem of “speckle” noise occurred when the video noises have adjacent correlation in the conventional temporal denoising system for videos.
1. A video denoising system based on noise correlation, which comprises: a frame memory, an inter-frame difference calculating unit, a motion detection unit, a filtering weight calculating unit, and a temporal filtering unit, characterized in that said system further comprises a unit for estimating correlation of noises of adjacent pixels, and a maximum filtering weight adjusting unit;
the unit for estimating correlation of noises of adjacent pixels calculates a noise correlation coefficient r h of horizontally adjacent pixels, a noise correlation coefficient r v of vertically adjacent pixels, a noise correlation coefficient r d of diagonally adjacent pixels according to an inter-frame difference d output from the inter-frame difference calculating unit and a motion probability m of each pixel output from the motion detection unit, and outputs the maximum value of r h , r v and r d as a noise correlation coefficient r;
the maximum filtering weight adjusting unit calculates and outputs a maximum weight Mw for performing temporal filtering according to the noise correlation coefficient r;
the filtering weight calculating unit calculates and outputs a filtering weight w for performing temporal filtering according to the motion probability m of each pixel and the maximum weight Mw for filtering.
2. The video denoising system based on noise correlation as claimed in claim 1 , characterized in that the unit for estimating correlation of noises of adjacent pixels calculates a noise correlation coefficient r by the equation of:
wherein c h , c v , c d are combination coefficients, r h is noise correlation coefficient of horizontally adjacent pixels, r v is a noise correlation coefficient of vertically adjacent pixels, r d is a noise correlation coefficient of diagonally adjacent pixels.
3. The video denoising system based on noise correlation as claimed in claim 1 , characterized in that the unit for estimating correlation of noises of adjacent pixels calculates a noise correlation coefficient r by the equation of:
r =max( r h ,r v ,r d )
wherein r h is a noise correlation coefficient of horizontally adjacent pixels, r v is a noise correlation coefficient of vertically adjacent pixels, r d is a noise correlation coefficient of diagonally adjacent pixels.
4. The video denoising system based on noise correlation as claimed in claim 1 , characterized in that the correlation coefficient r h of horizontally adjacent pixels, the correlation coefficient r v of vertically adjacent pixels, the correlation coefficient r d of diagonally adjacent pixels are calculated by the following equations:
r
h
=
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
,
j
+
1
)
*
m
(
i
,
j
)
*
m
(
i
,
j
+
1
)
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
,
j
)
*
m
(
i
,
j
)
*
m
(
i
,
j
+
1
)
r
v
=
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
d
(
i
,
j
)
*
d
(
i
+
1
,
j
)
*
m
(
i
,
j
)
*
m
(
i
+
1
,
j
)
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
,
j
)
*
m
(
i
,
j
)
*
m
(
i
+
1
,
j
)
r
d
=
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
+
1
,
j
+
1
)
*
m
(
i
,
j
)
*
m
(
i
+
1
,
j
+
1
)
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
,
j
)
*
m
(
i
,
j
)
*
m
(
i
+
1
,
j
)
wherein, d(i,j) is the inter-frame difference of the pixel at (i,j), m(i,j) is the motion probability of the pixel at (i,j), M and N are width and height of each frame of image of the video, respectively.
5. The video denoising system based on noise correlation as claimed in claim 4 , characterized in that a method for the maximum filtering weight adjusting unit to calculate the maximum weight Mw for filtering is as follows:
setting two thresholds r_T 1 and r_T 2 for the noise correlation r, wherein r_T 1 <r_T 2 ; setting two thresholds MW_V 1 and MW_V 0 for the maximum weight Mw for filtering, wherein MW_V 0 <MW_V 1 ;
when r≤r_T 1 , the maximum weight for filtering Mw is MW_V 1 , when r_T 1 <r<r_T 2 , the maximum weight for filtering Mw monotonically decreases from MW_V 1 to MW_V 0 as the noise correlation r increases, when r_T 2 ≤r, the maximum weight for filtering Mw is MW_V 0 .
6. The video denoising system based on noise correlation as claimed in claim 5 , the motion detection unit calculates the motion probability m of the pixel by the equation of:
m
(
i
,
j
)
=
{
1
if
M
A
D
(
i
,
j
)
≤
T
1
0
if
M
A
D
(
i
,
j
)
≥
T
2
*
T
2
-
MAD
(
i
,
j
)
T
2
-
T
1
else
wherein, MAD is a local Mean Absolute Difference feature of the pixel calculated according to the input inter-frame difference, T 1 and T 2 are soft thresholds for performing motion detection, and T 2 >T 1 .
7. The video denoising system based on noise correlation as claimed in claim 6 , characterized in that said filtering weight calculating unit calculates the filtering weight w for temporal filtering by the equation:
w
(
i
,
j
)
=
{
M
w
if
m
(
i
,
j
)
<
m_T
1
0
if
m
(
i
,
j
)
>
m_T
2
m
(
i
,
j
)
*
M
w
m_T
2
-
m_T
1
else
wherein m_T 1 and m_T 2 are thresholds for the motion probability, M w is the maximum filtering weight, and m is the motion probability of the pixel.
8. The video denoising system based on noise correlation as claimed in claim 7 , characterized in that the temporal filtering unit calculates the current filtered frame f^ t (i,j) by the equation:
f
t
⋀
(
i
,
j
)
=
w
(
i
,
j
)
*
f
t
-
1
⋀
(
i
,
j
)
+
f
t
(
i
,
j
)
1
+
w
(
i
,
j
)
wherein, f t is the current frame, f^ t−1 is the previous filtered frame in the frame memory, and W is the filtering weight for temporal filtering of the pixel.
9. The video denoising system based on noise correlation as claimed in claim 2 , characterized in that the noise correlation coefficient r h of horizontally adjacent pixels, the noise correlation coefficient r v of vertically adjacent pixels, the noise correlation coefficient r d of diagonally adjacent pixels are calculated by the following equations:
r
h
=
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
,
j
+
1
)
*
m
(
i
,
j
)
*
m
(
i
,
j
+
1
)
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
,
j
)
*
m
(
i
,
j
)
*
m
(
i
,
j
+
1
)
r
v
=
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
d
(
i
,
j
)
*
d
(
i
+
1
,
j
)
*
m
(
i
,
j
)
*
m
(
i
+
1
,
j
)
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
,
j
)
*
m
(
i
,
j
)
*
m
(
i
+
1
,
j
)
r
d
=
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
+
1
,
j
+
1
)
*
m
(
i
,
j
)
*
m
(
i
+
1
,
j
+
1
)
∑
i
=
0
i
<
M
∑
j
=
0
j
<
N
-
1
d
(
i
,
j
)
*
d
(
i
,
j
)
*
m
(
i
,
j
)
*
m
(
i
+
1
,
j
)
wherein, d(i,j) is the inter-frame difference of the pixel at (i,j), m(i,j) is the motion probability of the pixel at (i,j), M and N are width and height of each frame of image of the video, respectively.
10. The video denoising system based on noise correlation as claimed in claim 9 , characterized in that a method for the maximum filtering weight adjusting unit to calculate the maximum weight Mw for filtering is as follows:
setting two thresholds r_T 1 and r_T 2 for the noise correlation r, wherein r_T 1 <r_T 2 ; setting two thresholds MW_V 1 and MW_V 0 for the maximum weight Mw for filtering, wherein MW_V 0 <MW_V 1 ;
when r≤r_T 1 , the maximum weight for filtering Mw is MW_V 1 , when r_T 1 <r<r_T 2 , the maximum weight for filtering Mw monotonically decreases from MW_V 1 to MW_V 0 as the reliability r increases, when r_T 2 ≤r, the maximum weight for filtering Mw is MW_V 0 .
11. The video denoising system based on noise correlation as claimed in claim 10 , the motion detection unit calculates the motion probability m of the pixel by the equation of:
m
(
i
,
j
)
=
{
1
if
M
A
D
(
i
,
j
)
≤
T
1
0
if
M
A
D
(
i
,
j
)
≥
T
2
*
T
2
-
MAD
(
i
,
j
)
T
2
-
T
1
else
wherein, MAD is a local Mean Absolute Difference characteristic of the pixel calculated according to the input inter-frame difference, T 1 and T 2 are soft thresholds for performing motion detection, and T 2 >T 1 .
12. The video denoising system based on noise correlation as claimed in claim 11 , characterized in that said filtering weight calculating unit calculates the filtering weight w for temporal filtering by the equation:
w
(
i
,
j
)
=
{
M
w
if
m
(
i
,
j
)
<
m_T
1
0
if
m
(
i
,
j
)
>
m_T
2
m
(
i
,
j
)
*
M
w
m_T
2
-
m_T
1
else
wherein m_T 1 and m_T 2 are thresholds for the motion probability, M w is the maximum filtering weight, and m is the motion probability of the pixel.
13. The video denoising system based on noise correlation as claimed in claim 12 , characterized in that the temporal filtering unit calculates the present filtered frame f^ t (i,j) by the equation:
f
t
^
(
i
,
j
)
=
w
(
i
,
j
)
*
f
t
-
1
^
(
i
,
j
)
+
f
t
(
i
,
j
)
1
+
w
(
i
,
j
)
wherein, f t is the current frame, f^ t−1 is the previous filtered frame in the frame memory, and W is the filtering weight for temporal filtering of the pixel.