IP Library › Granted Patent US 12,513,342
Granted Patent B2
US 12,513,342 · App. 17/553,460 · Granted Dec 30, 2025

Apparatus, articles of manufacture, and methods for improved adaptive loop filtering in video encoding

Inventors: Ximin Zhang (San Jose, CA); Yi-Jen Chiu (San Jose, CA); Keith Rowe (Shingle Springs, CA)
Assignee: Intel Corporation
H04N19/82G06N20/00H04N19/107H04N19/117H04N19/147
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,513,342
App. No.
17/553,460
Granted
Dec 30, 2025
Kind
B2
Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed to improve video encoding. An example apparatus includes at least one memory, instructions, and processor circuitry to at least one of execute or instantiate the instructions to identify a first video frame as a critical video frame with respect to a previous video frame, generate filter coefficients of a video filter based on pixel data of the first video frame or pixel data of a previous critical video frame, and encode the first video frame with the filter coefficients.

Claims (93)

1 . An apparatus comprising:

interface circuitry instructions; and

at least one programmable circuit to be programmed based on the instructions to:

identify a first video frame as a critical video frame with respect to a previous video frame;

determine whether a latency associated with generation of filter coefficients of a video filter based on first pixel data of the first video frame satisfies a threshold;

based on a determination that the latency does not satisfy the threshold, generate the filter coefficients of the video filter based on the first pixel data of the first video frame;

based on a determination that the latency satisfies the threshold, generate the filter coefficients of the video filter based on second pixel data of a previous critical video frame; and

encode the first video frame with the filter coefficients.

2 . The apparatus of claim 1 , wherein one or more of the at least one programmable circuit is to restrict generation of video filters to be based on critical video frames.

3 . The apparatus of claim 1 , wherein one or more of the at least one programmable circuit is to:

select one or more types of the video filter to be applied to the first video frame;

identify a filter index corresponding to respective ones of the one or more types of the video filter;

encode the first video frame with the filter index; and

output the encoded first video frame.

4 . The apparatus of claim 1 , wherein the filter coefficients are first filter coefficients, and one or more of the at least one programmable circuit is to, based on a determination that a second video frame is not the critical video frame with respect to the previous video frame:

determine whether second filter coefficients associated with the previous critical video frame have been encoded in an adaptation parameter set data structure;

based on a determination that the second filter coefficients have been encoded in the adaptation parameter set data structure, select the second filter coefficients to be encoded in the second video frame; and

based on a determination that the second filter coefficients have not been encoded in the adaptation parameter set data structure, encode the second video frame with third filter coefficients stored in a decoded video frame buffer.

5 . The apparatus of claim 1 , wherein one or more of the at least one programmable circuit is to, based on a determination that a second video frame is not the critical video frame with respect to the previous video frame and a determination to turn off a chrominance adaptive loop filter:

generate a luminance adaptive loop filter;

identify a filter index corresponding to the luminance adaptive loop filter; and

encode the second video frame with the filter index.

6 . The apparatus of claim 1 , wherein one or more of the at least one programmable circuit is to, based on a determination that a second video frame is not the critical video frame with respect to the previous video frame and a determination to turn on a chrominance adaptive loop filter:

generate the chrominance adaptive loop filter, a luminance adaptive loop filter, and a cross-component adaptive loop filter;

identify a first filter index corresponding to the luminance adaptive loop filter, a second filter index corresponding to the chrominance adaptive loop filter, and a third filter index corresponding to the cross-component adaptive loop filter; and

encode the second video frame with the first filter index, the second filter index, and the third filter index.

7 . The apparatus of claim 1 , wherein one or more of the the at least one programmable circuit is to:

measure a temporal correlation difference between the first video frame and the previous video frame; and

identify the first video frame as the critical video frame based on the temporal correlation difference.

8 . The apparatus of claim 7 , wherein the threshold is a first threshold, and one or more of the at least one programmable circuit is to, based on a determination that the temporal correlation difference does not satisfy a second threshold:

determine a distance of the first video frame with respect to the previous video frame; and

identify the first video frame as the critical video frame based on the distance.

9 . An apparatus comprising:

means for identifying a first video frame as a critical video frame with respect to a previous video frame, the means for identifying to:

measure a temporal correlation difference between the first video frame and the previous video frame; and

identify the first video frame as the critical video frame based on the temporal correlation difference satisfying a threshold;

means for generating filter coefficients of a video filter based on first pixel data of the first video frame or second pixel data of a previous critical video frame; and

means for encoding the first video frame with the filter coefficients.

10 . The apparatus of claim 9 , wherein the means for generating is to restrict generation of video filters to be based on critical video frames.

11 . The apparatus of claim 9 , including:

means for determining whether a latency associated with generation of the filter coefficients based on the first pixel data of the first video frame satisfies a threshold; and

the means for generating to:

based on a determination that the latency does not satisfy the threshold, generate the filter coefficients based on the first pixel data; and

based on a determination that the latency satisfies the threshold, generate the filter coefficients based on the second pixel data.

12 . The apparatus of claim 9 , including:

means for selecting one or more types of the video filter to be applied to the first video frame; and

the means for encoding to:

identify a filter index corresponding to respective ones of the one or more types of the video filter;

encode the first video frame with the filter index; and

output the encoded first video frame.

13 . The apparatus of claim 9 , wherein the filter coefficients are first filter coefficients, and including:

means for selecting to, based on a determination that a second video frame is not the critical video frame with respect to the previous video frame:

determine whether second filter coefficients associated with the previous critical video frame have been encoded in an adaptation parameter set data structure; and

based on a determination that the second filter coefficients have been encoded in the adaptation parameter set data structure, select the second filter coefficients to be encoded in the second video frame; and

the means for encoding to, based on a determination that the second filter coefficients have not been encoded in the adaptation parameter set data structure, encode the second video frame with third filter coefficients stored in a decoded video frame buffer.

14 . The apparatus of claim 9 , wherein, based on a determination that a second video frame is not the critical video frame with respect to the previous video frame and a determination to turn off a chrominance adaptive loop filter:

the means for generating is to generate a luminance adaptive loop filter; and

the means for encoding is to:

identify a filter index corresponding to the luminance adaptive loop filter; and

encode the second video frame with the filter index.

15 . The apparatus of claim 9 , wherein, based on a determination that a second video frame is not the critical video frame with respect to the previous video frame and a determination to turn on a chrominance adaptive loop filter:

the means for generating is to generate the chrominance adaptive loop filter, a luminance adaptive loop filter, and a cross-component adaptive loop filter; and

the means for encoding is to:

identify a first filter index corresponding to the luminance adaptive loop filter, a second filter index corresponding to the chrominance adaptive loop filter, and a third filter index corresponding to the cross-component adaptive loop filter; and

encode the second video frame with the first filter index, the second filter index, and the third filter index.

16 . The apparatus of claim 9 , wherein the threshold is a first threshold, and, based on a determination that the temporal correlation difference does not satisfy the threshold, the means for identifying is to:

determine a distance of the first video frame with respect to the previous video frame; and

based on a determination that the distance satisfies a second threshold, identify the first video frame as the critical video frame.

17 . At least one non-transitory computer readable storage medium comprising instructions to cause at least one programmable circuit to at least:

determine a first video frame is not a critical video frame; and

based on a determination to turn on a chrominance adaptive loop filter:

generate the chrominance adaptive loop filter, a luminance adaptive loop filter, and a cross-component adaptive loop filter;

identify a first filter index corresponding to the luminance adaptive loop filter, a second filter index corresponding to the chrominance adaptive loop filter, and a third filter index corresponding to the cross-component adaptive loop filter; and

encode the first video frame with the first filter index, the second filter index, and the third filter index.

18 . The at least one non-transitory computer readable storage medium of claim 17 , wherein the instructions are to cause one or more of the at least one programmable circuit to limit generation of video to be based on critical video frames.

19 . The at least one non-transitory computer readable storage medium of claim 17 , wherein the instructions are to cause one or more of the at least one programmable circuit to:

identify a second video frame as the critical video frame;

determine whether a latency associated with generating filter coefficients of a video filter based on first pixel data of the second video frame satisfies a threshold;

based on determining that the latency does not satisfy the threshold, generate the filter coefficients based on the first pixel data; and

based on determining that the latency satisfies the threshold, generate the filter coefficients based on second pixel data of a previous critical video frame.

20 . The at least one non-transitory computer readable storage medium of claim 19 , wherein the instructions are to cause one or more of the at least one programmable circuit to:

select one or more types of the video filter to be applied to the second video frame;

identify a filter index corresponding to respective ones of the one or more types of the video filter;

encode the second video frame with the filter index; and

output the encoded second video frame.

21 . The at least one non-transitory computer readable storage medium of claim 17 , wherein the instructions are to cause one or more of the at least one programmable circuit to:

determine whether first filter coefficients associated with an intra-coded video frame have been encoded in an adaptation parameter set data structure;

based on a determination that the first filter coefficients have been encoded in the adaptation parameter set data structure, select the first filter coefficients to be encoded in the first video frame; and

based on a determination that the first filter coefficients have not been encoded in the adaptation parameter set data structure, encode the first video frame with second filter coefficients stored in a decoded video frame buffer.

22 . The at least one non-transitory computer readable storage medium of claim 17 , wherein the instructions are to cause one or more of the at least one programmable circuit to, based on a determination to turn off the chrominance adaptive loop filter:

generate the luminance adaptive loop filter;

identify the first filter index corresponding to the luminance adaptive loop filter; and

encode the first video frame with the first filter index.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 14, 2022
From: ZHANG, XIMIN; CHIU, YI-JEN; ROWE, KEITH
To: INTEL CORPORATION
Reel/Frame 059364/0093 →
Continuity (1)
Related Publication 20220109889A1 · Apr 7, 2022
References Cited (8)
US 20200404263A1 · Hu · 2020 [cited by examiner]
US 20210344922A1 · Zhang et al. · 2021 [cited by applicant]
US 20210344953A1 · Filippov et al. · 2021 [cited by applicant]
US 20210344964A1 · Drugeon et al. · 2021 [cited by applicant]
US 20220159249A1 · Taquet · 2022 [cited by examiner]
WO 2013059461 · 2013 [cited by applicant]
Karczewicz et al., “VVC In-Loop Filters,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, No. 10, Oct. 2021, 19 pages. [cited by applicant]
Bross et al., “Versatile Video Coding Editorial Refinements on Draft 10,” Joint Video Experts Team (JVET) U-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 20th Meeting, by teleconference, Oct. 7-16, 2020, 512 pages. [cited by applicant]