IP Library › Granted Patent US 12,739,368
Granted Patent B2
US 12,739,368 · App. 18/171,049 · Granted Sep 15, 2026

On adaptive loop filtering for video coding

Inventors: Hongbin Liu (Beijing, CN); Li Zhang (San Diego, CA); Kai Zhang (San Diego, CA); Hsiao Chiang Chuang (San Diego, CA); Zhipin Deng (Beijing, CN)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/117H04N19/132H04N19/176H04N19/70H04N19/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,739,368
App. No.
18/171,049
Granted
Sep 15, 2026
Kind
B2
Abstract

Devices, systems and methods for adaptive loop filtering are described. In an exemplary aspect, a method for video processing includes performing, for a current video block of a video, a filtering process that uses filter coefficients and comprises two or more operations with at least one intermediate result, applying a clipping operation to the at least one intermediate result, and performing, based on the at least one intermediate result, a conversion between the current video block and a bitstream representation of the video, wherein the at least one intermediate result is based on a weighted sum of the filter coefficients and differences between a current sample of the current video block and neighboring samples of the current sample.

Claims (45)

1 . A method of processing video data, comprising:

determining, for a conversion between a first video region of a video and a bitstream of the video, a first syntax element corresponding to the first video region is present in the bitstream, wherein the first syntax element indicates an index of a first adaptation parameter set referred to by the first video region; and

performing the conversion based on the determining,

wherein in response to the determining, a value of a second syntax element included in the first adaptation parameter set is 1, and wherein the second syntax element indicates whether to include a luma filter set in the first adaptation parameter set,

wherein for a sample of the first video region, a filter index is derived based on multiple sample differences in different directions, and a specific luma filter is selected, based on the filter index, from the luma filter set in the first adaption parameter set and fixed luma filters which are not included in the luma filter set,

wherein a maximum value of the filter index is indicated by N, and a number of luma filters in the luma filter set is smaller than or equal to N+1, and

wherein when the number of luma filters in the luma filter set is smaller than N+1, at least two luma filters with different filter indices have a same filter coefficient.

2 . The method of claim 1 , wherein the first video region is a video picture, and the first syntax element is present in a picture header syntax element set.

3 . The method of claim 1 , wherein the first video region is a video slice, and the first syntax element is present in a slice header syntax element set.

4 . The method of claim 1 , wherein the first video region includes multiple video coding tree blocks, and the video coding tree block is split into multiple M*M video block, where M is equal to 2 or 4, and

wherein a same filter index is applied for samples in a same M*M video block.

5 . The method of claim 4 , wherein the multiple sample differences in different directions are derived for every M*M video region based on 1:2 subsampling rate.

6 . The method of claim 1 , wherein the first adaptation parameter set includes an adaptive loop filtering adaptation parameter set.

7 . The method of claim 1 , wherein the conversion includes encoding a current video block into the bitstream.

8 . The method of claim 1 , wherein the conversion includes decoding a current video block from the bitstream.

9 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, for a conversion between a first video region of a video and a bitstream of the video, a first syntax element corresponding to the first video region is present in the bitstream, wherein the first syntax element indicates an index of a first adaptation parameter set referred to by the first video region; and

perform the conversion based on the determining;

wherein in response to the determining, a value of a second syntax element included in the first adaptation parameter set is 1, and wherein the second syntax element indicates whether to include a luma filter set in the first adaptation parameter set,

wherein for a sample of the first video region, a filter index is derived based on multiple sample differences in different directions, and a specific luma filter is selected, based on the filter index, from the luma filter set in the first adaption parameter set and fixed filters which are not included in the luma filter set,

wherein a maximum value of the filter index is indicated by N, and a number of luma filters in the luma filter set is smaller than or equal to N+1, and

wherein when the number of luma filters in the luma filter set is smaller than N+1, at least two luma filters with different filter indices have a same filter coefficient.

10 . The apparatus of claim 9 , wherein the first video region is a video picture, and the first syntax element is present in a picture header syntax element set.

11 . The apparatus of claim 9 , wherein the first video region is a video slice, and the first syntax element is present in a slice header syntax element set.

12 . The apparatus of claim 9 , wherein the first video region includes multiple video coding tree blocks, and the video coding tree block is split into multiple M*M video block, where M is equal to 2 or 4, and

wherein a same filter index is applied for samples in a same M*M video block.

13 . The apparatus of claim 9 , wherein the first adaptation parameter set includes an adaptive loop filtering adaptation parameter set.

14 . A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, for a conversion between a first video region of a video and a bitstream of the video, a first syntax element corresponding to the first video region is present in the bitstream, wherein the first syntax element indicates an index of a first adaptation parameter set referred to by the first video region; and

perform the conversion based on the determining;

wherein in response to the determining, a value of a second syntax element included in the first adaptation parameter set is 1, and wherein the second syntax element indicates whether to include a luma filter set in the first adaptation parameter set,

wherein for a sample of the first video region, a filter index is derived based on multiple sample differences in different directions, and a specific luma filter is selected, based on the filter index, from the luma filter set in the first adaption parameter set and fixed filters which are not included in the luma filter set,

wherein a maximum value of the filter index is indicated by N, and a number of luma filters in the luma filter set is smaller than or equal to N+1, and

wherein when the number of luma filters in the luma filter set is smaller than N+1, at least two luma filters with different filter indices have a same filter coefficient.

15 . A method for storing a bitstream of a video comprising:

determining, for a first video region of the video, a first syntax element corresponding to the first video region is present in the bitstream, wherein the first syntax element indicates an index of a first adaptation parameter set referred to by the first video region;

generating the bitstream based on the determining; and

storing the bitstream in a non-transitory computer-readable recording medium,

wherein in response to the determining, a value of a second syntax element included in the first adaptation parameter set is 1, and wherein the second syntax element indicates whether to include a luma filter set in the first adaptation parameter set,

wherein for a sample of the first video region, a filter index is derived based on multiple sample differences in different directions, and a specific luma filter is selected, based on the filter index, from the luma filter set in the first adaption parameter set and fixed filters which are not included in the luma filter set,

wherein a maximum value of the filter index is indicated by N, and a number of luma filters in the luma filter set is smaller than or equal to N+1, and

wherein when the number of luma filters in the luma filter set is smaller than N+1, at least two luma filters with different filter indices have a same filter coefficient.

16 . The medium of claim 14 , wherein first video region is a video picture, and the first syntax element is present in a picture header syntax element set;

wherein the first video region is a video slice, and the first syntax element is present in a slice header syntax element set; and

wherein the first adaptation parameter set includes an adaptive loop filtering adaptation parameter set.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 17, 2023
From: ZHANG, LI; ZHANG, KAI; CHUANG, HSIAO CHIANG
To: BYTEDANCE INC.
Reel/Frame 062735/0100 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 17, 2023
From: LIU, HONGBIN; DENG, ZHIPIN
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 062735/0203 →
Priority Claims (1)
WO PCT/CN2019/082855 · Apr 16, 2019 · international
Continuity (3)
Continuation 17399839 · Aug 11, 2021
Continuation PCTCN2020085075 · Apr 16, 2020
Related Publication 20230217021A1 · Jul 6, 2023
References Cited (92)
US 8929462B2 · Wang · 2015 [cited by applicant]
US 9591331B2 · Chong · 2017 [cited by applicant]
US 9813719B2 · Wang · 2017 [cited by applicant]
US 10057574B2 · Li · 2018 [cited by applicant]
US 10271052B2 · Zhang · 2019 [cited by applicant]
US 10277909B2 · Ye · 2019 [cited by applicant]
US 10404988B2 · Ye · 2019 [cited by applicant]
US 10506230B2 · Zhang · 2019 [cited by applicant]
US 10582213B2 · Li · 2020 [cited by applicant]
US 10694202B2 · Zhang · 2020 [cited by applicant]
US 10694204B2 · Chen · 2020 [cited by applicant]
US 10721469B2 · Zhang · 2020 [cited by applicant]
US 10764576B2 · Li · 2020 [cited by applicant]
US 10778974B2 · Karczewicz · 2020 [cited by applicant]
US 10855985B2 · Zhang · 2020 [cited by applicant]
US 20070058713A1 · Shen · 2007 [cited by applicant]
US 20090116558A1 · Chen · 2009 [cited by applicant]
US 20130107973A1 · Wang · 2013 [cited by applicant]
US 20130259118A1 · Fu · 2013 [cited by examiner]
US 20150016551A1 · Esenlik · 2015 [cited by applicant]
US 20150043641A1 · Gamei · 2015 [cited by examiner]
US 20150049821A1 · Chen · 2015 [cited by applicant]
US 20160234492A1 · Li · 2016 [cited by applicant]
US 20170237981A1 · Karczewicz · 2017 [cited by applicant]
US 20170238020A1 · Karczewicz · 2017 [cited by applicant]
US 20180041778A1 · Zhang · 2018 [cited by applicant]
US 20180041779A1 · Zhang · 2018 [cited by applicant]
US 20180192050A1 · Zhang · 2018 [cited by applicant]
US 20180270480A1 · Zhang · 2018 [cited by applicant]
US 20180288441A1 · Zhang · 2018 [cited by applicant]
US 20180302621A1 · Fu · 2018 [cited by applicant]
US 20180359486A1 · Lai · 2018 [cited by applicant]
US 20190014315A1 · Karczewicz · 2019 [cited by applicant]
US 20190238845A1 · Zhang · 2019 [cited by applicant]
US 20190306502A1 · Gadde · 2019 [cited by applicant]
US 20200177910A1 · Li · 2020 [cited by applicant]
US 20200267381A1 · Vanam · 2020 [cited by applicant]
US 20200322632A1 · Hanhart · 2020 [cited by applicant]
US 20200359016A1 · Li · 2020 [cited by applicant]
US 20200359017A1 · Li · 2020 [cited by applicant]
US 20200359018A1 · Li · 2020 [cited by applicant]
US 20200413038A1 · Zhang · 2020 [cited by applicant]
US 20210051329A1 · Ko · 2021 [cited by examiner]
US 20210329272A1 · Rusanovskyy · 2021 [cited by examiner]
US 20240364877A1 · Taquet · 2024 [cited by examiner]
CN 102857746A · 2013 [cited by applicant]
CN 102939760A · 2013 [cited by applicant]
CN 103051890A · 2013 [cited by applicant]
CN 103096054A · 2013 [cited by applicant]
CN 103563380A · 2014 [cited by applicant]
CN 103959777A · 2014 [cited by applicant]
CN 104378632A · 2015 [cited by applicant]
CN 104735450A · 2015 [cited by applicant]
CN 107211154A · 2017 [cited by applicant]
CN 109031227A · 2018 [cited by applicant]
CN 109076219A · 2018 [cited by applicant]
CN 113767623B · 2021 [cited by applicant]
EP 3297282A1 · 2018 [cited by applicant]
JP 2013524678A · 2013 [cited by applicant]
JP 2015080002A · 2015 [cited by applicant]
JP 2017513335A · 2017 [cited by applicant]
JP 2018509074A · 2018 [cited by applicant]
JP 2020017970A · 2020 [cited by applicant]
WO 2012134204A2 · 2012 [cited by applicant]
WO 2013145174A1 · 2013 [cited by applicant]
Document: JVET-L0147, Lim, S., et al., “CE2: Subsampled Laplacian Calculation (Test 6.1, 6.2, 6.3 and 6.4),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct… [cited by applicant]
Document: JVET-N0242, Taquet, J., et al., “CE5 : Results of Tests CE5-3.1 to CE5-3.4 on Non-Linear Adaptive Filter,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva… [cited by applicant]
Document: JVET-N0415-v2, Hu, N., et al., “CE5: Coding Tree Block Based Adaptive Loop Filter( CE5-4),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-2… [cited by applicant]
Document: JVET-K1001-v6, Bross, B., et al., “Versatile Video Coding (Draft 2),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018, 141 pages. [cited by applicant]
Document: JVET-M0427-v2, Li, T., et al., “CE12: Mapping Functions (Test CE 12-1 and CE 12-2),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Marrakech, MA, Jan. 9-18, 20… [cited by applicant]
Document: JVET-L1001-v9, Bross, B., et al., “Versatile Video Coding (Draft 3),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018, 235 pages. [cited by applicant]
Document: JVET-N0236-r5, Luo, J., et al., “CE2-Related : Prediction Refinement with Optical Flow for Affine Mode,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, … [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video High efficiency video coding,” ITU-T and ISO/IEC, “High efficiency video coding”, Rec. ITU-T H.265 | ISO/IEC 23… [cited by applicant]
Document: JVET-G1001-v1, Chen, J., et al., “Algorithm description of Joint Exploration Test Model 7 (JEM7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT,… [cited by applicant]
Karczewicz, M., et al., “Geometry transformation based adaptive in-loop filter,” Picture Coding Symposium (PCS), IEEE, 2016, 5 pages. [cited by applicant]
Document: JVET-M0385, Taquet, J., et al., “Non-Linear Adaptive Loop Filter,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Jan. 18, 2019, 6 pages. [cited by applicant]
Document: JVET-N0025, Seregin, V., et al., “CE5: Summary Report on Adaptive Loop Filter,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Mar. 27, 2019, 13 pages. [cited by applicant]
Document: JVET-K0382-v3, Karczewicz, M., et al., CE2-related: CTU Based Adaptive Loop Filtering, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 11th Meeting: Ljubljana, SI, Jul. 10-18,… [cited by applicant]
Document: JCTVC-10157, Tsai, C., et al., “AHG6: Baseline Options for ALF,” Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC 29/WG 11 and ITU-T SG 16, Apr. 23, 2012, 13 pages. [cited by applicant]
Document: JVET-K0164-v2, Lim, S., et al., “CE2: Subsampled Sum-Modified-Laplacian (test 4.1.1),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 11th Meeting, Ljubljana, SI, Jul. 10-18,… [cited by applicant]
Foreign Communication From a Related Counterpart Application, European Application No. 20791885.5, Extended European Search Report dated May 13, 2022, 9 pages. [cited by applicant]
Final Office Action dated Apr. 19, 2022, 10 pages, U.S. Appl. No. 17/399,839, filed Aug. 11, 2021. [cited by applicant]
Non-Final Office Action dated Dec. 15, 2021, 16 pages, U.S. Appl. No. 17/399,839, filed Aug. 11, 2021. [cited by applicant]
Notice of Allowance dated Nov. 3, 2022, 12 pages, U.S. Appl. No. 17/399,839, filed Aug. 11, 2021. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2020/085068 dated Jun. 30, 2020, 9 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2020/085075 dated Jul. 15, 2020, 12 pages. [cited by applicant]
Document: JVET-D1001-v3, Chen, J., et al., “Algorithm Description of Joint Exploration Test Model 4,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 4th Meeting: Chengdu, CN, Oct.… [cited by applicant]
Document JVET-J0013-v3, Kang, J., et al., “Description of SDR video coding technology proposal by ETRI and Sejong University,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 10th Meet… [cited by applicant]
Document: JVET-K0164-v1, Lim, S-C. “CE2: Subsampled sum-modified-Laplacian (Test 4.1.1),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and 1S0/IEC JTC 1/SC 29/WG 11 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018, … [cited by applicant]
European Office Action from European Application No. 20791885.5 dated Jun. 7, 2024, 7 pages. [cited by applicant]
Bross B., et al., “Versatile Video Coding (Draft 2),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018, Document: JVET-K1001-v7, 280 Pages. [cited by applicant]
Notice of Refusal for Japanese Application No. 2021-559943, mailed Oct. 29, 2024, 32 pages. [cited by applicant]