Filtering artifacts in images with 3D spatio-temporal fuzzy filters
View Patent ↗A method filters pixels in a sequence of images. Each image in the sequence is partitioned into blocks of pixels, and the images are processed sequentially. The energy is determined for each block of pixels in each image. The energy of each block is based on variances of intensities of the pixels in the sequence of images. A 3D fuzzy filter is applied to each current pixel in each current block during the sequential processing. The 3D fuzzy filter considers the energy of the block, and the intensities of pixels spatially adjacent and temporally adjacent to the current pixel to remove blocking and ringing artifacts.
1. A method for filtering pixels in a sequence of images, in which each image in the sequence is partitioned into blocks of pixels, and the images are processed sequentially, comprising the steps of:
defining a set of temporally and spatially adjacent pixels to a current pixel with dimension (T − +T + +1)×(2M+1)×(2N+1), where the set includes pixels from T − previous images, a current image, and T + next images, within a spatial boundary of (2M+1)×(2N+1) centered around the current pixel;
determining an energy for each block of pixels in each image, in which the energy of each block is based on variances of intensities of the set of spatially and temporally adjacent pixels in the sequence of images; and
calculating, for each pixel in the set of spatially and temporally adjacent pixels, a fuzzy filter weight, where the fuzzy filter weight is associated with each current pixel in each current block and a particular pixel in the set of spatially and temporally adjacent pixels, and the fuzzy filter weight is determined by evaluating a one-dimensional Gaussian membership function at two pixel intensity values including the current pixel and the particular pixel in the set of spatially and temporally adjacent pixels with the energy of the block as a spread parameter; and
applying a fuzzy filter to each current pixel in each current block during the sequential processing, where the output of the fuzzy filter is defined as a weighted average of all pixels in the set of spatially and temporally adjacent pixels with associated fuzzy filter weights, to remove blocking and ringing artifacts, wherein the steps of the method are performed by a computer processor.
2. The method of claim 1 , in which the temporally adjacent blocks are colocated spatially.
3. The method of claim 1 , in which the temporally adjacent blocks are shifted spatially according to a motion vector of the current block.
4. The method of claim 1 , in which an output of the fuzzy filter is
∑
Δt
=
-
T
-
T
+
∑
Δ
m
=
-
M
M
∑
Δ
n
=
-
N
N
I
(
t
,
m
,
n
)
×
μ
G
(
I
(
t
,
m
,
n
)
,
I
(
t
+
Δ
t
,
m
+
Δ
m
,
n
+
Δ
n
)
)
∑
Δ
t
=
-
T
-
T
+
∑
Δ
m
=
-
M
M
∑
Δ
n
=
-
N
N
μ
G
(
I
(
t
,
m
,
n
)
,
I
(
t
+
Δ
t
,
m
+
Δ
m
,
n
+
Δ
n
)
)
,
in which I(t, m, n) is the current pixel identified by coordinates t, m, and n, wherein t is a temporal dimension, and m, n are the spatial dimensions of the coordinates, Δ indicates coordinates of the set of temporally and spatially adjacent pixels to the current pixel, μ G (a,b) is the one-dimensional membership function defined by μ G (a,b)=e −(a−b) 2 /2σ 2 evaluated at two pixel intensity values a and b within the set of pixels and σ is the spread parameter.
5. The method of claim 1 , in which the fuzzy filter is applied to motion objects in the sequence of images.
6. The method of claim 1 , in which T − and T + are different.
7. The method of claim 1 , in which the fuzzy filter is applied to chrominance components.
8. The method of claim 1 , in which the fuzzy filter is applied to luminance components.