IP Library › Granted Patent US 12,464,128
Granted Patent B2
US 12,464,128 · App. 18/298,787 · Granted Nov 4, 2025

Virtual boundary processing for CCSAO, bilateral filter and adaptive loop filter for video coding

Inventors: Venkata Meher Satchit Anand Kotra (Munich, DE); Nan Hu (San Diego, CA); Vadim Seregin (San Diego, CA); Marta Karczewicz (San Diego, CA)
Assignee: QUALCOMM INCORPORATED
H04N19/132H04N19/117H04N19/167H04N19/176H04N19/80H04N19/85
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,464,128
App. No.
18/298,787
Granted
Nov 4, 2025
Kind
B2
Abstract

An example device for decoding video data includes a memory configured to store video data; and one or more processors implemented in circuitry and configured to: decode a current block of the video data to form a decoded block; determine that a current sample of the decoded block neighbors a sample along a virtual boundary in the decoded block and neighbors one or more samples that are not along any virtual boundary in the decoded block; compute band information for cross component sample adaptive offset (CCSAO) for the current sample using at least one of the one or more samples that are not along any virtual boundary in the decoded block and without using the sample along the virtual boundary; and perform CCSAO on the current sample using the band information.

Claims (46)

1 . A method of decoding video data, the method comprising:

decoding a current block of video data to form a decoded block;

determining that a current sample of the decoded block neighbors a sample along a virtual boundary in the decoded block and neighbors one or more samples that are not along any virtual boundary in the decoded block;

selecting a selected sample from the one or more samples that are not along any virtual boundary in the decoded block;

computing a band classification for cross component sample adaptive offset (CCSAO) for the current sample using the selected sample and without using any other neighboring samples to the current sample; and

performing CCSAO on the current sample using the band classification.

2 . The method of claim 1 , further comprising disabling CCSAO for the sample along the virtual boundary.

3 . The method of claim 1 , further comprising performing bilateral filtering (BIF) on the current sample using a padding value that replaces the value of the sample along the virtual boundary, the padding value comprising the value of one of the one or more samples that are not along any virtual boundary in the decoded block.

4 . The method of claim 3 , further comprising replacing values of samples along or beyond the virtual boundaries in the decoded block and within a filtering region of the sample with padding values determined from the one or more samples that are not along any virtual boundary in the decoded block.

5 . The method of claim 1 , further comprising encoding the current block prior to decoding the current block.

6 . A device for decoding video data, the device comprising:

a memory configured to store video data; and

one or more processors implemented in circuitry and configured to:

decode a current block of the video data to form a decoded block;

determine that a current sample of the decoded block neighbors a sample along a virtual boundary in the decoded block and neighbors one or more samples that are not along any virtual boundary in the decoded block;

select a selected sample from the one or more samples that are not along any virtual boundary in the decoded block;

compute a band classification for cross component sample adaptive offset (CCSAO) for the current sample using the selected sample and without using any other neighboring samples to the current sample; and

perform CCSAO on the current sample using the band classification.

7 . The device of claim 6 , wherein the one or more processors are further configured to disable CCSAO for the sample along the virtual boundary.

8 . The device of claim 6 , wherein the one or more processors are further configured to perform bilateral filtering (BIF) on the current sample using a padding value that replaces the value of the sample along the virtual boundary, the padding value comprising the value of one of the one or more samples that are not along any virtual boundary in the decoded block.

9 . The device of claim 8 , wherein the one or more processors are further configured to replace values of samples along or beyond the virtual boundaries in the decoded block and within a filtering region of the sample with padding values determined from the one or more samples that are not along any virtual boundary in the decoded block.

10 . The device of claim 6 , wherein the one or more processors are further configured to encode the current block prior to decoding the current block.

11 . The device of claim 6 , further comprising a display configured to display decoded video data including the decoded block.

12 . The device of claim 6 , wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

13 . A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:

decode a current block of video data to form a decoded block;

determine that a current sample of the decoded block neighbors a sample along a virtual boundary in the decoded block and neighbors one or more samples that are not along any virtual boundary in the decoded block;

select a selected sample from the one or more samples that are not along any virtual boundary in the decoded block;

compute a band classification for cross component sample adaptive offset (CCSAO) for the current sample using the selected sample and without using any other neighboring samples to the current sample; and

perform CCSAO on the current sample using the band classification.

14 . The non-transitory computer-readable storage medium of claim 13 , further comprising instructions that cause the processor to disable CCSAO for the sample along the virtual boundary.

15 . The non-transitory computer-readable storage medium of claim 13 , further comprising instructions that cause the processor to perform bilateral filtering (BIF) on the current sample using a padding value that replaces the value of the sample along the virtual boundary, the padding value comprising the value of one of the one or more samples that are not along any virtual boundary in the decoded block.

16 . The non-transitory computer-readable storage medium of claim 15 , further comprising instructions that cause the processor to replace values of samples along or beyond the virtual boundaries in the decoded block and within a filtering region of the sample with padding values determined from the one or more samples that are not along any virtual boundary in the decoded block.

17 . The non-transitory computer-readable storage medium of claim 13 , further comprising instructions that cause the processor to encode the current block prior to decoding the current block.

18 . A device for decoding video data, the device comprising:

means for decoding a current block of video data to form a decoded block;

means for determining that a sample of the decoded block neighbors a sample along a virtual boundary in the decoded block and neighbors one or more samples that are not along any virtual boundary in the decoded block;

means for selecting a selected sample from the one or more samples that are not along any virtual boundary in the decoded block;

means for computing a band classification for cross component sample adaptive offset (CCSAO) for the sample using the selected sample and without using any other neighboring samples to the current sample; and

means for performing CCSAO on the sample using the band classification.

19 . The method of claim 1 , wherein determining that the current sample of the decoded block neighbors the sample along the virtual boundary in the decoded block and neighbors the one or more samples that are not along any virtual boundary in the decoded block comprises processing each neighboring sample of the current sample of the decoded block to determine that the current sample of the decoded block neighbors the sample along the virtual boundary in the decoded block and neighbors the one or more samples that are not along any virtual boundary in the decoded block.

20 . The method of claim 1 , wherein the virtual boundary comprises one of a vertical virtual boundary defined by a constant horizontal coordinate value “X” or a horizontal virtual boundary defined by a constant vertical coordinate value “Y”.

21 . The device of claim 6 , wherein to determine that the current sample of the decoded block neighbors the sample along the virtual boundary in the decoded block and neighbors the one or more samples that are not along any virtual boundary in the decoded block, the processing system is configured to process each neighboring sample of the current sample of the decoded block to determine that the current sample of the decoded block neighbors the sample along the virtual boundary in the decoded block and neighbors the one or more samples that are not along any virtual boundary in the decoded block.

22 . The device of claim 6 , wherein the virtual boundary comprises one of a vertical virtual boundary defined by a constant horizontal coordinate value “X” or a horizontal virtual boundary defined by a constant vertical coordinate value “Y”.

23 . The non-transitory computer-readable storage medium of claim 13 , wherein the instructions that cause the processor to determine that the current sample of the decoded block neighbors the sample along the virtual boundary in the decoded block and neighbors the one or more samples that are not along any virtual boundary in the decoded block comprise instructions that cause the processor to process each neighboring sample of the current sample of the decoded block to determine that the current sample of the decoded block neighbors the sample along the virtual boundary in the decoded block and neighbors the one or more samples that are not along any virtual boundary in the decoded block, and wherein the virtual boundary comprises one of a vertical virtual boundary defined by a constant horizontal coordinate value “X” or a horizontal virtual boundary defined by a constant vertical coordinate value “Y”.

24 . The device of claim 18 , wherein the means for determining that the current sample of the decoded block neighbors the sample along the virtual boundary in the decoded block and neighbors the one or more samples that are not along any virtual boundary in the decoded block comprises means for processing each neighboring sample of the current sample of the decoded block to determine that the current sample of the decoded block neighbors the sample along the virtual boundary in the decoded block and neighbors the one or more samples that are not along any virtual boundary in the decoded block, and wherein the virtual boundary comprises one of a vertical virtual boundary defined by a constant horizontal coordinate value “X” or a horizontal virtual boundary defined by a constant vertical coordinate value “Y”.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 12, 2023
From: KOTRA, VENKATA MEHER SATCHIT ANAND; HU, NAN; SEREGIN, VADIM; KARCZEWICZ, MARTA
To: QUALCOMM INCORPORATED
Reel/Frame 063625/0105 →
Continuity (2)
Provisional Application 63362932 · Apr 13, 2022
Related Publication 20230336734A1 · Oct 19, 2023
References Cited (22)
US 20210152841A1 · Hu et al. · 2021 [cited by applicant]
WO 2022066783A1 · 2022 [cited by applicant]
Bross B., et al., “Versatile Video Coding (Draft 10)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 131, MPEG Meeting, 19th Meeting, by Teleconference, Jun. 22-Jul. 1, 2020, Jun. 29… [cited by applicant]
Coban M., et al., “Algorithm Description of Enhanced Compression Model 3 (ECM 3)”, JVET-X2025-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 23rd Meeting, by teleconference, Jul. 7-16, … [cited by applicant]
“Files ECM-3.0 ECM/ECM GitLab”, Retrieved on Jul. 12, 2023, pp. 1-4. [cited by applicant]
International Search Report and Written Opinion—PCT/US2023/018277—ISAEPO—Jul. 7, 2023, 14 Pages. [cited by applicant]
ITU-T H.265: “Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video”, High Efficiency Video Coding, The International Telecommunication Union, Jun. 2019, 696 Pages. [cited by applicant]
ITU-T H.266: “Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video”, Versatile Video Coding, The International Telecommunication Union, Aug. 2020, 516 pages. [cited by applicant]
Karczewicz M., et al., “Common Test Conditions and Evaluation Procedures for Enhanced Compression Tool Testing”, JVET-Y2017-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 25th Meeting, … [cited by applicant]
Karczewicz M., et al., “Common Test Conditions and Evaluation Procedures for Enhanced Compression Tool Testing”, JVET-W2017-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, JVET-X2017-v1,… [cited by applicant]
Kotra A.M., et al., “AHG12: Virtual Boundary Processing for the New In-Loop filter tools in ECM”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 26th Meeting, by teleconference, Apr. 20-29,… [cited by applicant]
Kuo C-W., et al., “AHG12: Cross-Component Sample Adaptive Offset”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 22nd Meeting, by teleconference, Apr. 20-28, 2021, No. JVET-V0153-v2, pp. 1… [cited by applicant]
Kuo C-W., et al., “EE2-5.1: Cross-Component Sample Adaptive Offset”, 23, JVET Meeting, Jul. 7, 2021-Jul. 16, 2021, Teleconference, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-TSG.16), No. JVET-W0… [cited by applicant]
Lin S-Y., “CE13: In-Loop Filters Disabled Across Face Discontinuities (Test 1.1.a and Test1.1.b)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-1… [cited by applicant]
Lin S-Y., “CE-13 Related: Loop Filter Disabled Across Virtual Boundaries”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, No. JVET-M0892-… [cited by applicant]
Meng X., et al., “CCALF Virtual Boundary Issue for 4:4:4 and 4:2:2 Format”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 18th Meeting: by teleconference, Apr. 15-24, 2020, No. JVET-… [cited by applicant]
Poirier T., et al., “AHG7: GDR in ECM-4.0”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 26th Meeting, by teleconference, Apr. 20-29, 2022, No. JVET-Z0141-v4, pp. 1-3. [cited by applicant]
Strom J., et al., “CE1-related: Multiplication-free Bilateral Loop Filter”, JVET-N0493_v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, p… [cited by applicant]
Strom J., et al., “CE5-3.1 Combination of Bilateral Filter and SAO”, JVET-P0073_v3, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 16th Meeting: Geneva, CH, Oct. 1-11, 2019, pp. 1-18. [cited by applicant]
Strom J., et al., “EE2: Bilateral Filter in VTM, EE2 and VVenC”, 22, JVET Meeting, Apr. 20, 2021-Apr. 28, 2021, Teleconference, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-TSG.16), No. JVET-V0094… [cited by applicant]
Wang L., et al., “AHG 7: GDR without encoder constraints for ECM”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 25th Meeting, by teleconference, Jan. 12-21, 2022, No. JVET-Y0163-r2, pp. 1… [cited by applicant]
Wikipedia: “Sobel-Operator”, Retrieved on Jul. 13, 2023, pp. 1-3. [cited by applicant]