IP Library Granted Patent US 11,641,488
Granted Patent B2
US 11,641,488 · App. 16/760,974 · Granted May 2, 2023

Methods for simplifying adaptive loop filter in video coding

Inventors: Rahul Vanam (San Diego, CA); Yuwen He (San Diego, CA); Yan Ye (San Diego, CA)
Assignee: VID SCALE, Inc.
H04N19/105G06K9/628G06K9/6228G06K9/6268H04N19/117H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,641,488
App. No.
16/760,974
Granted
May 2, 2023
Kind
B2
Abstract

Systems, methods and instrumentalities are disclosed for adaptively selecting an adaptive loop filter (ALF) procedure for a frame based on which temporal layer the frame is in. ALF procedures may vary in computational complexity. One or more frames including the current frame may be in a temporal layer of a coding scheme. The decoder may determine the current frame's temporal layer level within the coding scheme. The decoder may select an ALF procedure based on the current frame's temporal layer level. If the current frame's temporal layer level is higher within the coding scheme than some other temporal layer levels, an ALF procedure that is less computationally complex may be selected for the current frame. Then the decoder may perform the selected ALF procedure on the current frame.

Claims (68)

1. A video decoding device comprising:

a processor configured to:

select, from a plurality of pixels of an 8×8 pixel window, a subset of pixels for classifying a video block associated with the 8×8 pixel window, wherein the subset of pixels is selected by skipping, in a diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window;

obtain a sum of diagonal variations using the subset of pixels that is selected by skipping, in the diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window;

determine a diagonal gradient based on the sum of diagonal variations;

classify the video block for adaptive loop filtering (ALF) based on the determined diagonal gradient;

perform ALF on the video block based on the classification of the video block; and

decode a frame comprising the video block.

2. The video decoding device of claim 1 , wherein the subset of pixels is selected from the plurality of pixels by skipping at least one pixel in a vertical direction of the 8×8 pixel window and at least one pixel in a horizontal direction of the 8×8 pixel window.

3. The video decoding device of claim 1 , wherein the processor is further configured to receive at least one of a first indication that the subset of pixels is to be selected for block classification, or a second indication of how the subset of pixels is to be selected.

4. The video decoding device of claim 1 , wherein the obtained sum of diagonal variations is associated with a first diagonal direction, the determined diagonal gradient is a first diagonal gradient associated with the first diagonal direction, and the processor is further configured to:

obtain a sum of diagonal variations associated with a second diagonal direction; and

determine a second diagonal gradient associated with the second diagonal direction based on the sum of diagonal variations associated with the second diagonal direction, wherein the video block is classified further based on the second diagonal gradient.

5. The video decoding device of claim 1 , wherein the processor is further configured to:

determine a temporal layer level of the frame within a coding scheme; and

determine whether to select the subset of pixels for classifying the video block based on the temporal layer level of the frame, wherein, based on a condition that the temporal layer level is the highest within the coding scheme, the subset of pixels is determined to be selected.

6. The video decoding device of claim 1 , further comprising a memory operatively coupled to the processor.

7. The video decoding device of claim 1 , wherein the processor is further configured to:

determine a 4×4 block, wherein the video block comprises the 4×4 block;

identify a neighboring pixel of the 4×4 block in the subset of pixels selected from the 8×8 pixel window; and

obtain a diagonal variation based on the identified neighboring pixel.

8. The video decoding device of claim 1 , wherein the diagonal gradient is a first diagonal gradient, and the processor is further configured to obtain a vertical gradient, a horizontal gradient, and a second diagonal gradient using the subset of pixels that is selected by skipping, in the diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window, wherein the video block is classified further based on the obtained vertical gradient, the obtained horizontal gradient, and the obtained second diagonal gradient.

9. A video decoding method, comprising:

selecting, from a plurality of pixels of an 8×8 pixel window, a subset of pixels for classifying a video block associated with the 8×8 pixel window, wherein the subset of pixels is selected by skipping, in a diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window;

obtaining a sum of diagonal variations using the subset of pixels that is selected by skipping, in the diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window;

determining a diagonal gradient based on the sum of diagonal variations;

classifying the video block for adaptive loop filtering (ALF) based on the determined diagonal gradient;

performing ALF on the video block based on the classification of the video block; and

decoding a frame comprising the video block.

10. The video decoding method of claim 9 , wherein the subset of pixels is selected from the plurality of pixels by skipping at least one pixel in a vertical direction of the 8×8 pixel window and at least one pixel in a horizontal direction of the 8×8 pixel window.

11. The video decoding method of claim 9 , wherein the obtained sum of diagonal variations is associated with a first diagonal direction, and the determined diagonal gradient is a first diagonal gradient associated with the first diagonal direction, and the method further comprises:

obtaining a sum of diagonal variations associated with a second diagonal direction; and

determining a second diagonal gradient associated with the second diagonal direction based on the sum of diagonal variations associated with the second diagonal direction, wherein the video block is classified further based on the second diagonal gradient.

12. The video decoding method of claim 9 , wherein the method further comprises:

determining a 4×4 block, wherein the video block comprises the 4×4 block;

identifying a neighboring pixel of the 4×4 block in the subset of pixels selected from the 8×8 pixel window; and

obtaining a diagonal variation based on the identified neighboring pixel.

13. A video encoding device comprising:

a processor configured to:

select, from a plurality of pixels of an 8×8 pixel window, a subset of pixels for classifying a video block associated with the 8×8 pixel window, wherein the subset of pixels is selected by skipping, in a diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window;

obtain a sum of diagonal variations using the subset of pixels that is selected by skipping, in the diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window;

determine a diagonal gradient based on the sum of diagonal variations;

classify the video block for adaptive loop filtering (ALF) based on the determined diagonal gradient;

perform ALF on the video block based on the classification of the video block; and

generate a residual based on the video block.

14. The video encoding device of claim 13 , wherein the processor is further configured to:

determine a 4×4 block, wherein the video block comprises the 4×4 block;

identify a neighboring pixel of the 4×4 block in the subset of pixels selected from the 8×8 pixel window; and

obtain a diagonal variation based on the identified neighboring pixel.

15. The video encoding device of claim 13 , wherein the diagonal gradient is a first diagonal gradient, and the processor is further configured to obtain a vertical gradient, a horizontal gradient, and a second diagonal gradient using the subset of pixels that is selected by skipping, in the diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window, wherein the video block is classified further based on the obtained vertical gradient, the obtained horizontal gradient, and the obtained second diagonal gradient.

16. The video encoding device of claim 13 , wherein the obtained sum of diagonal variations is associated with a first diagonal direction, and the determined diagonal gradient is a first diagonal gradient associated with the first diagonal direction, and the processor is further configured to:

obtain a sum of diagonal variations associated with a second diagonal direction; and

determine a second diagonal gradient associated with the second diagonal direction based on the sum of diagonal variations associated with the second diagonal direction, wherein the video block is classified further based on the second diagonal gradient.

17. A video encoding method, comprising:

selecting, from a plurality of pixels of an 8×8 pixel window, a subset of pixels for classifying a video block associated with the 8×8 pixel window, wherein the subset of pixels is selected by skipping, in a diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window;

obtaining a sum of diagonal variations using the subset of pixels that is selected by skipping, in the diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window;

determining a diagonal gradient based on the sum of diagonal variations;

classifying the video block for adaptive loop filtering (ALF) based on the determined diagonal gradient;

performing ALF on the video block based on the classification of the video block; and

generating a residual based on the video block.

18. The video encoding method of claim 17 , wherein the diagonal gradient is a first diagonal gradient, and the method further comprises obtaining a vertical gradient, a horizontal gradient, and a second diagonal gradient using the subset of pixels that is selected by skipping, in the diagonal direction, alternate lines of the plurality of pixels of the 8×8 pixel window, wherein the video block is classified further based on the obtained vertical gradient, the obtained horizontal gradient, and the obtained second diagonal gradient.

19. The video encoding method of claim 17 , wherein the obtained sum of diagonal variations is associated with a first diagonal direction, and the determined diagonal gradient is a first diagonal gradient associated with the first diagonal direction, and the method further comprises:

obtaining a sum of diagonal variations associated with a second diagonal direction; and

determining a second diagonal gradient associated with the second diagonal direction based on the sum of diagonal variations associated with the second diagonal direction, wherein the video block is classified further based on the second diagonal gradient.

20. The video encoding method of claim 17 , further comprising:

determining a 4×4 block, wherein the video block comprises the 4×4 block;

identifying a neighboring pixel of the 4×4 block in the subset of pixels selected from the 8×8 pixel window; and

obtain a diagonal variation based on the identified neighboring pixel.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2024
From: VANAM, RAHUL; HE, YUWEN; YE, YAN
To: VID SCALE, INC.
Reel/Frame 068572/0114 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2024
From: VID SCALE, INC.
To: INTERDIGITAL VC HOLDINGS, INC.
Reel/Frame 068284/0031 →