Multi-metric filtering
A filter unit of a video encoder or video decoder can determine a first metric for a group of pixels within a block of pixels, determine a second metric for the group of pixels, determine a filter based on the first metric and the second metric, and generate a filtered image by applying the filter to the group of pixels. The first metric and second metric can be an activity metric and a direction metric, respectively, or can be other metrics such as an edge metric, horizontal activity metric, vertical activity metric, or diagonal activity metric.
1. A method of video coding, the method comprising:
determining a first metric for a group of pixels within a block of pixels, wherein the first metric is determined based on comparing pixel values of the group of pixels to pixel values of the block of pixels;
determining a second metric for the group of pixels, wherein determining the second metric comprises determining a direction of the group of pixels from a set of directions, wherein the set of directions includes a horizontal direction, a vertical direction, a 45-degree direction, and a 135-degree direction;
based on the first metric and the second metric, determining a filter; and
generating a filtered image by applying the filter to the group of pixels.
2. The method of claim 1 , wherein the first metric comprises an activity metric.
3. The method of claim 2 , wherein the activity metric comprises a sum-modified Laplacian value, wherein the sum-modified Laplacian value comprises var(i,j) defined according to the following equation:
var
(
i
,
j
)
=
∑
k
=
-
K
K
∑
l
=
-
L
L
2
R
(
i
+
k
,
j
+
l
)
-
R
(
i
+
k
-
1
,
j
+
l
)
-
R
(
i
+
k
+
1
,
j
+
l
+
2
R
(
i
+
k
,
j
+
l
)
-
R
(
i
+
k
,
j
+
l
-
1
)
-
R
(
i
+
k
,
j
+
l
+
1
)
wherein k represents a value of a summation from −K to K and l represents a value of a summation from −L to L for a two-dimensional window that spans from −K to K and −L to L, wherein i and j represent pixel coordinates of the pixel data, R(i,j) represents a given pixel value at coordinates i and j, and var(i,j) is the activity metric.
4. The method of claim 1 , wherein determining the second first metric for the group of pixels comprises:
comparing at least one pixel of the group of pixels to a first neighboring pixel within the block of pixels;
in response to a pixel value of the at least one pixel being greater than a pixel value of the first neighboring pixel, incrementing an edge metric variable; and
in response to the pixel value of the at least one pixel being less than the pixel value of the first neighboring pixel, decrementing the edge metric variable.
5. The method of claim 1 , wherein the first metric is determined based on comparing a pixel value of at least one pixel of the group of pixels to a pixel value of a left neighboring pixel within the block of pixels and a pixel value of a right neighboring pixel within the block of pixels.
6. The method of claim 1 , wherein the first metric is determined based on comparing a pixel value of at least one pixel of the group of pixels to a pixel value of an upper neighboring pixel within the block of pixels and a pixel value of lower neighboring pixel within the block of pixels.
7. The method of claim 2 , wherein the activity metric is determined based on a sum of the measure of horizontal activity and the measure of vertical activity.
8. The method of claim 1 , wherein the group of pixels comprises a 4×4 block of pixels.
9. The method of claim 1 , wherein the group of pixels comprises one pixel.
10. The method of claim 1 , wherein the method of video coding implements a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple filters.
11. The method of claim 1 , wherein the method is performed by a video coding device comprising a video encoder.
12. The method of claim 1 , wherein the method is performed by a video coding device comprising a video decoder.
13. A video coding device comprising:
a filter unit configured to:
determine a first metric for a group of pixels within a block of pixels, wherein the first metric is determined based on comparing pixel values of the group of pixels to pixel values of the block of pixels;
determine a second metric for the group of pixels, wherein the second metric comprises a direction selected from a set of directions, wherein the set of directions includes a horizontal direction, a vertical direction, a 45-degree direction, and a 135-degree direction;
determine a filter based on the first metric and the second metric; and
generate a filtered image by applying the filter to the group of pixels; and
a memory configured to store a filtered result of the filter unit.
14. The video coding device of claim 13 , wherein the first metric comprises an activity metric.
15. The video coding device of claim 14 , wherein the activity metric comprises a sum-modified Laplacian value, and wherein the sum-modified Laplacian value comprises var(i,j) defined according to the following equation:
var
(
i
,
j
)
=
∑
k
=
-
K
K
∑
l
=
-
L
L
2
R
(
i
+
k
,
j
+
l
)
-
R
(
i
+
k
-
1
,
j
+
l
)
-
R
(
i
+
k
+
1
,
j
+
l
+
2
R
(
i
+
k
,
j
+
l
)
-
R
(
i
+
k
,
j
+
l
-
1
)
-
R
(
i
+
k
,
j
+
l
+
1
)
wherein k represents a value of a summation from −K to K and l represents a value of a summation from −L to L for a two-dimensional window that spans from −K to K and −L to L, wherein i and j represent pixel coordinates of the pixel data, R(i,j) represents a given pixel value at coordinates i and j, and var(i,j) is the activity metric.
16. The video coding device of claim 13 , wherein the filter unit is further configured to determine the first metric for the group of pixels by:
comparing at least one pixel of the group of pixels to a first neighboring pixel within the block of pixels;
in response to a pixel value of the at least one pixel being greater than a pixel value of the first neighboring pixel, incrementing an edge metric variable; and
in response to the pixel value of the at least one pixel being less than the pixel value of the first neighboring pixel, decrementing the edge metric variable.
17. The video coding device of claim 13 , wherein the filter unit determines the first metric based on comparing a pixel value of at least one pixel of the group of pixels to a pixel value of a left neighboring pixel within the block of pixels and a pixel value of a right neighboring pixel within the block of pixels.
18. The video coding device of claim 13 , wherein the filter unit determines the first metric based on comparing a pixel value of at least one pixel of the group of pixels to a pixel value of an upper neighboring pixel within the block of pixels and a pixel value of lower neighboring pixel within the block of pixels.
19. The video coding device of claim 14 , wherein the activity metric is determined based on a sum of the measure of horizontal activity and the measure of vertical activity.
20. The video coding device of claim 13 , wherein the group of pixels comprises a 4×4 block of pixels.
21. The video coding device of claim 13 , wherein the group of pixels comprises one pixel.
22. The video coding device of claim 13 , wherein the video coding device implements a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple filters.
23. The video coding device of claim 13 , wherein the filter unit is a part of a video encoding device.
24. The video coding device of claim 13 , wherein the filter unit is a part of a video decoding device.
25. An apparatus comprising:
means for determining a first metric for a group of pixels within a block of pixels, wherein the means for determining the first metric comprises means for determining the first metric based on comparing pixel values of the group of pixels to pixel values of the block of pixels;
means for determining a second metric for the group of pixels, wherein the second metric comprises a direction selected from a set of directions, wherein the set of directions includes a horizontal direction, a vertical direction, a 45-degree direction, and a 135-degree direction;
means for determining a filter based on the first metric and the second metric; and
means for generating a filtered image by applying the filter to the group of pixels.
26. The apparatus of claim 25 , wherein the first metric comprises an activity metric.
27. The apparatus of claim 26 , wherein the activity metric comprises a sum-modified Laplacian value, and wherein the sum-modified Laplacian value comprises var(i,j) defined according to the following equation:
var
(
i
,
j
)
=
∑
k
=
-
K
K
∑
l
=
-
L
L
2
R
(
i
+
k
,
j
+
l
)
-
R
(
i
+
k
-
1
,
j
+
l
)
-
R
(
i
+
k
+
1
,
j
+
l
+
2
R
(
i
+
k
,
j
+
l
)
-
R
(
i
+
k
,
j
+
l
-
1
)
-
R
(
i
+
k
,
j
+
l
+
1
)
wherein k represents a value of a summation from −K to K and l represents a value of a summation from −L to L for a two-dimensional window that spans from −K to K and −L to L, wherein i and j represent pixel coordinates of the pixel data, R(i,j) represents a given pixel value at coordinates i and j, and var(i,j) is the activity metric.
28. The apparatus of claim 25 , wherein the means for determining the first metric for the group of pixels comprises:
means for comparing at least one pixel of the group of pixels to a first neighboring pixel within the block of pixels;
means for incrementing an edge metric variable in response to a pixel value of the at least one pixel being greater than a pixel value of the first neighboring pixel; and
means for decrementing the edge metric variable in response to the pixel value of the at least one pixel being less than the pixel value of the first neighboring pixel.
29. The apparatus of claim 25 , wherein the first metric is determined based on comparing a pixel value of at least one pixel of the group of pixels to a pixel value of a left neighboring pixel within the block of pixels and a pixel value of a right neighboring pixel within the block of pixels.
30. The apparatus of claim 25 , wherein the first metric is determined based on comparing a pixel value of at least one pixel of the group of pixels to a pixel value of an upper neighboring pixel within the block of pixels and a pixel value of lower neighboring pixel within the block of pixels.
31. The apparatus of claim 26 , wherein the activity metric is determined based on a sum of the measure of horizontal activity and the measure of vertical activity.
32. The apparatus of claim 25 , wherein the group of pixels comprises a 4×4 block of pixels.
33. The apparatus of claim 25 , wherein the group of pixels comprises one pixel.
34. The apparatus of claim 25 , wherein the apparatus implements a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple filters.
35. The apparatus of claim 25 , wherein the apparatus comprises a video coding device comprising a video encoder.
36. The apparatus of claim 25 , wherein the apparatus comprises a video coding device comprising a video decoder.
37. A non-transitory, computer-readable storage medium having stored thereon instructions that when executed cause one or more processors to:
determine a first metric for a group of pixels within a block of pixels, wherein the first metric is determined based on comparing pixel values of the group of pixels to pixel values of the block of pixels;
determine a second metric for the group of pixels, wherein the second metric comprises a direction selected from a set of directions, wherein the set of directions includes a horizontal direction, a vertical direction, a 45-degree direction, and a 135-degree direction;
determine a filter based on the first metric and the second metric; and
generate a filtered image by applying the filter to the group of pixels.
38. The non-transitory, computer-readable storage medium of claim 37 , wherein the first metric comprises an activity metric.
39. The non-transitory, computer-readable storage medium of claim 38 , wherein the activity metric comprises a sum-modified Laplacian value, and where in the sum-modified Laplacian value comprises var(i,j) defined according to the following equation:
var
(
i
,
j
)
=
∑
k
=
-
K
K
∑
l
=
-
L
L
2
R
(
i
+
k
,
j
+
l
)
-
R
(
i
+
k
-
1
,
j
+
l
)
-
R
(
i
+
k
+
1
,
j
+
l
+
2
R
(
i
+
k
,
j
+
l
)
-
R
(
i
+
k
,
j
+
l
-
1
)
-
R
(
i
+
k
,
j
+
l
+
1
)
wherein k represents a value of a summation from −K to K and l represents a value of a summation from −L to L for a two-dimensional window that spans from −K to K and −L to L, wherein i and j represent pixel coordinates of the pixel data, R(i,j) represents a given pixel value at coordinates i and j, and var(i,j) is the activity metric.
40. The non-transitory, computer-readable storage medium of claim 37 , storing further instructions that when executed cause the one or more processors to determining the first metric for the group of pixels by:
comparing at least one pixel of the group of pixels to a first neighboring pixel within the block of pixels;
incrementing an edge metric variable in response to a pixel value of the at least one pixel being greater than a pixel value of the first neighboring pixel; and
decrementing the edge metric variable in response to the pixel value of the at least one pixel being less than the pixel value of the first neighboring pixel.
41. The non-transitory, computer-readable storage medium of claim 37 , wherein the first metric is determined based on comparing a pixel value of at least one pixel of the group of pixels to a pixel value of a left neighboring pixel within the block of pixels and a pixel value of a right neighboring pixel within the block of pixels.
42. The non-transitory, computer-readable storage medium of claim 37 , wherein the first metric is determined based on comparing a pixel value of at least one pixel of the group of pixels to a pixel value of an upper neighboring pixel within the block of pixels and a pixel value of lower neighboring pixel within the block of pixels.
43. The non-transitory, computer-readable storage medium of claim 38 , wherein the activity metric is determined based on a sum of the measure of horizontal activity and the measure of vertical activity.
44. The non-transitory, computer-readable storage medium of claim 37 , wherein the group of pixels comprises a 4×4 block of pixels.
45. The non-transitory, computer-readable storage medium of claim 37 , wherein the group of pixels comprises one pixel.
46. The non-transitory, computer-readable storage medium of claim 37 , wherein the one or more processors implement a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple filters.
47. The non-transitory, computer-readable storage medium of claim 37 , wherein the one or more processors perform video encoding.
48. The non-transitory, computer-readable storage medium of claim 37 , wherein the one or more processors perform video decoding.