IP Library › Granted Patent US 12,316,859
Granted Patent B2
US 12,316,859 · App. 17/553,449 · Granted May 27, 2025

Apparatus, articles of manufacture, and methods for clipping index identification in adaptive loop filtering for video encoding

Inventors: Ximin Zhang (San Jose, CA); Yi-Jen Chiu (San Jose, CA); Keith Rowe (Shingle Springs, CA)
Assignee: Intel Corporation
H04N19/147G06N20/00H04N19/107H04N19/117H04N19/127
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,316,859
App. No.
17/553,449
Granted
May 27, 2025
Kind
B2
Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed to improve video encoding. An example apparatus includes at least one memory, instructions, and processor circuitry to generate a pool of clipping index set candidates by executing a machine learning model, select a clipping index set from the pool of the clipping index set candidates based on a rate distortion cost associated with the clipping index set, the clipping index set including clipping coefficients, and filter a video frame based on the clipping coefficients.

Claims (82)

1. An apparatus to improve video encoding, the apparatus comprising:

at least one memory;

instructions; and

processor circuitry to at least one of execute or instantiate the instructions to:

generate a pool of clipping index set candidates by executing a machine learning model;

select a clipping index set from the pool of the clipping index set candidates based on a rate distortion cost associated with the clipping index set, the clipping index set including clipping coefficients; and

filter a video frame based on the clipping coefficients.

2. The apparatus of claim 1 , wherein the processor circuitry is to at least one of execute or instantiate the instructions to:

determine a spatial variance of the video frame; and

in response to determining that the video frame is a smooth video frame based on the spatial variance, disable clipping for the video frame.

3. The apparatus of claim 2 , wherein the processor circuitry is to at least one of execute or instantiate the instructions to:

determine whether the spatial variance satisfies a threshold;

identify the video frame as a smooth video frame based on the spatial variance not satisfying the threshold; and

determine that the video frame is not a smooth video frame based on the spatial variance satisfying the threshold.

4. The apparatus of claim 1 , wherein the processor circuitry is to at least one of execute or instantiate the instructions to adjust a priority of the clipping index set based on the selection of the clipping index set for encoding the video frame.

5. The apparatus of claim 4 , wherein the processor circuitry is to at least one of execute or instantiate the instructions to adjust the priority of the clipping index set by at least one of increasing the priority of the clipping index set or decreasing a respective priority of one or more of the pool of the clipping index set candidates not selected.

6. The apparatus of claim 1 , wherein the processor circuitry is to at least one of execute or instantiate the instructions to select the clipping index set in response to determining that an encoder is configured to process numerous clipping index sets.

7. The apparatus of claim 1 , wherein the clipping index set is a first clipping index set, the pool of the clipping index set candidates include the first clipping index set and a second clipping index set, and the processor circuitry is to at least one of execute or instantiate the instructions to:

generate a number of filters to filter the video frame based on the first clipping index set;

adjust one or more first values of the clipping coefficients of the first clipping index set to one or more second values;

determine a first rate distortion cost associated with the filters based on the one or more second values; and

in response to determining that the first rate distortion cost is less than a second rate distortion cost associated with the second clipping index set, select the first clipping index set with the one or more second values.

8. The apparatus of claim 1 , wherein the processor circuitry is to at least one of execute or instantiate the instructions to:

instantiate a machine learning model based on machine learning model parameters;

execute the machine learning model to output a machine learning output; and

in response to the machine learning output satisfying a threshold, deploy the machine learning model to:

generate the pool of the clipping index set candidates;

assign a priority to respective ones of the pool of the clipping index set candidates; and

store associations of the priorities and respective ones of the pool of the clipping index set candidates.

9. An apparatus to improve video encoding, the apparatus comprising:

means for generating a pool of clipping index set candidates by executing a machine learning model;

means for selecting a clipping index set from the pool of the clipping index set candidates based on a rate distortion cost associated with the clipping index set, the clipping index set including clipping coefficients; and

means for filtering a video frame based on the clipping coefficients.

10. The apparatus of claim 9 , further including:

means for determining a spatial variance of the video frame; and

the means for selecting is to, in response to determining that the video frame is a smooth video frame based on the spatial variance, disable clipping for the video frame.

11. The apparatus of claim 10 , wherein the means for determining is to:

determine whether the spatial variance satisfies a threshold;

identify the video frame as a smooth video frame based on the spatial variance not satisfying the threshold; and

determine that the video frame is not a smooth video frame based on the spatial variance satisfying the threshold.

12. The apparatus of claim 9 , further including means for adjusting to adjust a priority of the clipping index set based on the selection of the clipping index set for encoding the video frame.

13. The apparatus of claim 12 , wherein the means for adjusting is to adjust the priority of the clipping index set by at least one of increasing the priority of the clipping index set or decreasing a respective priority of one or more of the pool of the clipping index set candidates not selected.

14. The apparatus of claim 9 , wherein the means for selecting is to select the clipping index set in response to determining that an encoder is configured to process numerous clipping index sets.

15. The apparatus of claim 9 , wherein the clipping index set is a first clipping index set, the pool of the clipping index set candidates include the first clipping index set and a second clipping index set, and wherein:

the means for filtering is to generate a number of filters to filter the video frame based on the first clipping index set;

means for adjusting to:

adjust one or more first values of the clipping coefficients of the first clipping index set to one or more second values; and

determine a first rate distortion cost associated with the filters based on the one or more second values; and

the means for selecting to, in response to determining that the first rate distortion cost is less than a second rate distortion cost associated with the second clipping index set, select the first clipping index set with the one or more second values.

16. The apparatus of claim 9 , further including means for executing a machine learning model, the means for executing to:

instantiate the machine learning model based on machine learning model parameters;

execute the machine learning model to output a machine learning output; and

in response to the machine learning output satisfying a threshold, deploy the machine learning model to:

generate the pool of the clipping index set candidates;

assign a priority to respective ones of the pool of the clipping index set candidates; and

store associations of the priorities and respective ones of the pool of the clipping index set candidates.

17. At least one non-transitory computer readable storage medium comprising instructions that, when executed, cause processor circuitry to at least:

generate a pool of clipping index set candidates by executing a machine learning model;

select a clipping index set from the pool of the clipping index set candidates based on a rate distortion cost associated with the clipping index set, the clipping index set including clipping coefficients; and

filter a video frame based on the clipping coefficients.

18. The at least one non-transitory computer readable storage medium of claim 17 , wherein the instructions, when executed, cause the processor circuitry to:

determine a spatial variance of the video frame; and

in response to determining that the video frame is a smooth video frame based on the spatial variance, disable clipping for the video frame.

19. The at least one non-transitory computer readable storage medium of claim 18 , wherein the instructions, when executed, cause the processor circuitry to:

determine whether the spatial variance satisfies a threshold;

identify the video frame as a smooth video frame based on the spatial variance not satisfying the threshold; and

determine that the video frame is not a smooth video frame based on the spatial variance satisfying the threshold.

20. The at least one non-transitory computer readable storage medium of claim 17 , wherein the instructions, when executed, cause the processor circuitry to adjust a priority of the clipping index set based on the selection of the clipping index set for encoding the video frame.

21. The at least one non-transitory computer readable storage medium of claim 20 , wherein the instructions, when executed, cause the processor circuitry to adjust the priority of the clipping index set based on at least one of an increase of the priority of the clipping index set or a decrease of a respective priority of one or more of the pool of the clipping index set candidates not selected.

22. The at least one non-transitory computer readable storage medium of claim 17 , wherein the instructions, when executed, cause the processor circuitry to select the clipping index set in response to a determination that an encoder is configured to process numerous clipping index sets.

23. The at least one non-transitory computer readable storage medium of claim 17 , wherein the clipping index set is a first clipping index set, the clipping index set candidates include the first clipping index set and a second clipping index set, and the instructions, when executed, cause the processor circuitry to:

generate a number of filters to filter the video frame based on the first clipping index set;

adjust one or more first values of the clipping coefficients of the first clipping index set to one or more second values;

determine a first rate distortion cost associated with the filters based on the one or more second values; and

in response to a determination that the first rate distortion cost is less than a second rate distortion cost associated with the second clipping index set, select the first clipping index set with the one or more second values.

24. The at least one non-transitory computer readable storage medium of claim 17 , wherein the instructions, when executed, cause the processor circuitry to at least one of execute or instantiate the instructions to:

instantiate a machine learning model based on machine learning model parameters;

execute the machine learning model to output a machine learning output; and

in response to a determination that the machine learning output satisfies a threshold, execute the machine learning model to:

generate the pool of the clipping index set candidates;

assign a priority to respective ones of the pool of the clipping index set candidates; and

store associations of the priorities and respective ones of the pool of the clipping index set candidates.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2022
From: ZHANG, XIMIN; CHIU, YI-JEN; ROWE, KEITH
To: INTEL CORPORATION
Reel/Frame 059135/0561 →
Continuity (1)
Related Publication 20220109853A1 · Apr 7, 2022
References Cited (6)
US 20210344922A1 · Zhang et al. · 2021 [cited by applicant]
US 20210344953A1 · Filippov et al. · 2021 [cited by applicant]
US 20210344964A1 · Drugeon et al. · 2021 [cited by applicant]
WO 2013059461 · 2013 [cited by applicant]
Karczewicz et al., “VVC In-Loop Filters,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, No. 10, Oct. 2021, 19 pages. [cited by applicant]
Bross et al., “Versatile Video Coding Editorial Refinements on Draft 10,” Joint Video Experts Team (JVET) U-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 20th Meeting, by teleconference, Oct. 7-16, 2020, 512 pages. [cited by applicant]
Cited By (1)
US 12,549,718