Spatio-temporal denoising of video content based on confidence indexes
The disclosure relates to a method for processing the noise of a stream comprising a series of images, comprising spatial filtering of a current image I n in order to obtain a spatially filtered image S n ; determining a motion vector for each block B of pixels of the image S n , from a single image S n-1 ; calculating a confidence index α n [B] for each block B of image I n , from the motion vector determined for B; constructing a temporal filter from the calculated confidence indices α n [B] and applying the temporal filter to the current image I n in order to obtain a temporally filtered image T n ; spatial filtering of the current image I n in order to obtain a spatially filtered image D n where D n can be equal to S n , and obtaining a temporally and spatially filtered image R n by applying a function combining image T n and image D n .
1. A method, implemented by computer means, for processing the noise of a digital video stream comprising a series of successive images, the method comprising:
spatial filtering to denoise a current image I n of the series in order to obtain a spatially filtered image S n , n being a natural number;
determining a motion vector for each block B of pixels of the image S n , from a single image S n-1 which precedes image S n in the series of digital images;
calculating a confidence index α n [B] for each block B of image I n , from the motion vector determined for B, where α n [B] is representative of a temporal correlation between B and the block corresponding to B in image S n-1 ;
constructing a temporal filter from the calculated confidence indices α n [B] and applying the temporal filter to the current image I n in order to obtain a temporally filtered image T n ;
spatial filtering to denoise the current image I n of the series in order to obtain a spatially filtered image D n where D n can be equal to S n ,
obtaining a temporally and spatially filtered image R n by applying a function combining the temporally filtered image T n and the spatially filtered image D n ,
wherein the confidence index α n [B] is calculated from a parameter P[B] evaluating the correlation between block B of image D n and block B* of image D n-1 , and
wherein the confidence index α n [B] is expressed as follows:
∝
n
[
B
]
=
P
[
B
]
*
clip
(
s
+
v
*
∝
n
-
1
[
B
*
]
,
0
,
1
)
with
clip
(
x
,
min
,
max
)
=
{
min
if
x
<
min
max
if
x
>
max
else
x
where:
s denotes an offset
v a learning speed
∝ n-1 [B*] the confidence index of block B* of image S n-1 .
2. The method according to claim 1 , wherein a block B of the temporally filtered image T n is written as T n [B] and is expressed as a linear combination of T n-1 [B*] and I n [B] where:
T n_1 [B*] designates the block B* corresponding to block B in the temporally filtered image T n-1 .
3. The method according to claim 2 , wherein T n [B] is expressed in the following form:
T n [ B ]=∝ n [ B ]* T n-1 [ B *]+(1−∝ n [ B ])* I n [ B ].
4. The method according to claim 3 , wherein for n=0, the following expression is satisfied for all blocks B of image T 0 :
T 0 [ B ]= I 0 [ B ] and ∝ 0 [ B ]=0.
5. The method according to claim 1 , wherein 0≤P[B]≤1 and 0≤α n [B]≤1.
6. The method according to claim 1 , wherein the temporally and spatially filtered image R n is expressed in the form of a linear combination of T n [B] and D n [B], where D n [B] denotes a block B of spatially filtered image D n .
7. The method according to claim 6 , wherein the temporally and spatially filtered image R n is expressed in the following form:
R n [ B ]=∝ n [ B ]* T n [ B ]+(1−∝ n [ B ])* D n [ B ].
8. A non-transitory, computer-readable medium having stored thereon computer-executable instructions for carrying out the method of claim 1 .
9. A device for processing noise in a digital video stream comprising a series of digital images, the device comprising:
a processor configured to perform the following operations:
spatial filtering to denoise a current image I n of the series in order to obtain a spatially filtered image S n , n being a natural number;
determining a motion vector for each block B of pixels of the image S n , from a single image S n-1 which precedes image S n in the series of digital images;
calculating a confidence index α n [B] for each block B of image I n , from the motion vector determined for B, where α n [B] is representative of a temporal correlation between B and the block corresponding to B in image S n-1 ;
constructing a temporal filter from the calculated confidence indices α n [B] and applying the temporal filter to the current image I n in order to obtain a temporally filtered image T n ;
spatial filtering to denoise the current image I n of the series in order to obtain a spatially filtered image D n where D n can be equal to S n ,
obtaining a temporally and spatially filtered image R n by applying a function combining the temporally filtered image T n and the spatially filtered image D n ,
wherein the confidence index α n [B] is calculated from a parameter P[B] evaluating the correlation between block B of image D n and block B* of image D n-1 , and
wherein the confidence index α n [B] is expressed as follows:
∝
n
[
B
]
=
P
[
B
]
*
clip
(
s
+
v
*
∝
n
-
1
[
B
*
]
,
0
,
1
)
with
clip
(
x
,
min
,
max
)
=
{
min
if
x
<
min
max
if
x
>
max
else
x
where:
s denotes an offset
v a learning speed
∝ n-1 [B*] the confidence index of block B* of image S n-1 .