IP Library › Granted Patent US 12,348,786
Granted Patent B2
US 12,348,786 · App. 17/979,027 · Granted Jul 1, 2025

Enhanced cross component sample offset filtering system for video coding and decoding

Inventors: Yixin Du (Milpitas, CA); Alexandros Tourapis (Los Gatos, CA); Yunfei Zheng (Santa Clara, CA); Jun Xin (San Jose, CA); Alican Nalci (Cupertino, CA); Mei T. Guo (San Jose, CA); Yeqing Wu (Cupertino, CA); Hsi-Jung Wu (San Jose, CA)
Assignee: APPLE INC.
H04N19/82H04N19/186H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,348,786
App. No.
17/979,027
Granted
Jul 1, 2025
Kind
B2
Abstract

A cross-component based filtering system is disclosed for video coders and decoders. The filtering system may include a filter having an input for a filter offset and an input for samples reconstructed from coded video data representing a native component of source video on which the filter operates. The offset may be generated at least in part from a sample classifier that classifies samples reconstructed from coded video data representing a color component of the source video orthogonal to the native component according to sample intensity.

Claims (43)

1. A filtering system, comprising:

a filter having an input for a filter offset and an input for samples reconstructed from coded video data representing a first color component of source video in a multi-color color space on which the filter operates and having an output for filtered samples,

a intensity classifier having an input for samples reconstructed from coded video data representing a second color component of the source video different from the first color component in the multi-color color space, and having an output for data representing an intensity classification of the samples input thereto,

an offset calculator having a first input for the intensity classification data from the sample classifier and having an output for a filter offset that is input to the filter.

2. The filtering system of claim 1 , wherein the intensity classification data from the intensity classifier represents a classification of a reconstructed sample into one of a plurality of sample intensity bands.

3. The filtering system of claim 2 , wherein the sample intensity bands are defined by signaling provided by a video encoder that generated the coded video data.

4. The filtering system of claim 3 , wherein the signaling identifies a number of the sample intensity bands.

5. The filtering system of claim 3 , wherein the signaling identifies the sample intensity bands as uniformly spaced.

6. The filtering system of claim 3 , wherein the signaling identifies breakpoints among the sample intensity bands.

7. The filtering system of claim 1 , wherein the intensity classification data from the sample classifier represents a classification of a block of reconstructed samples into one of a plurality of sample intensity bands.

8. The filtering system of claim 1 , wherein

the reconstructed samples are received from an output of a second filter of a video coding system, and

the intensity classification data is derived from a filtering parameter of the second filter.

9. The filtering system of claim 1 , wherein the offset calculator is an N-way lookup table having a way corresponding to each of a number of candidate intensity classifications.

10. The filtering system of claim 9 , further comprising an index calculator having an input for the reconstructed samples representing the second color component and having an output for an index identifier, wherein the offset calculator has a second input for the index identifier output from the index calculator.

11. The filtering system of claim 9 , wherein at least one entry of the lookup table is provided by a video encoder that generated the coded video data.

12. The filtering system of claim 1 , wherein the offset calculator is a computer that calculates the filter offset according to a parametric model, and parameters of the parametric model are provided by a video encoder that generated the coded video data.

13. A method, comprising:

classifying an intensity of first input samples reconstructed from coded video data representing a first color component of source video in a multi-color color space different from a second color component in the multi-color color space that will be filtered,

estimating a filter offset based on the classified intensity of the first input samples, and

filtering second input samples reconstructed from the coded video data representing the second color component of the source video according to the estimated filter offset.

14. The method of claim 13 , wherein the classifying classifies the reconstructed samples individually into one of a plurality of sample intensity bands.

15. The method of claim 14 , wherein the sample intensity bands are defined by signaling provided by a video encoder that generated the coded video data.

16. The method of claim 15 , wherein the signaling identifies a number of the sample intensity bands.

17. The method of claim 15 , wherein the signaling identifies the sample intensity bands as uniformly spaced.

18. The method of claim 15 , wherein the signaling identifies breakpoints among the sample intensity bands.

19. The method of claim 13 , wherein the classifying classifies the reconstructed samples into one of a plurality of sample intensity bands on a pixel-block by pixel-block basis.

20. The method of claim 13 , wherein the estimating comprising reading an offset value from a lookup table using the classified intensity as an index.

21. The method of claim 20 , wherein the lookup table is an N-way lookup table having a way corresponding to each of a number of candidate intensity classifications.

22. The method of claim 20 , wherein at least one entry of the lookup table is provided in signaling by a video encoder that generated the coded video data.

23. A computer readable medium having program instructions stored thereon that, when executed by a processing device, cause the processing device to perform a method, comprising:

classifying an intensity of first input samples reconstructed from coded video data representing a first color component of source video in a multi-color color space different from a second color component that will be filtered,

estimating a filter offset based on the classified intensity of the first input samples, and

filtering second input samples reconstructed from the coded video data representing the second color component of the source video according to the estimated filter offset.

24. The computer readable medium of claim 23 , wherein the classifying classifies the reconstructed samples individually into one of a plurality of sample intensity bands.

25. The computer readable medium of claim 24 , wherein the sample intensity bands are defined by signaling provided by a video encoder that generated the coded video data.

26. The computer readable medium of claim 25 , wherein the signaling identifies a number of the sample intensity bands.

27. The computer readable medium of claim 25 , wherein the signaling identifies the sample intensity bands as uniformly spaced.

28. The computer readable medium of claim 25 , wherein the signaling identifies breakpoints among the sample intensity bands.

29. The computer readable medium of claim 23 , wherein the classifying classifies the reconstructed samples into one of a plurality of sample intensity bands on a pixel-block by pixel-block basis.

30. The computer readable medium of claim 23 , wherein the estimating comprising reading an offset value from a lookup table using the classified intensity as an index.

31. The computer readable medium of claim 30 , wherein the lookup table is an N-way lookup table having a way corresponding to each of a number of candidate intensity classifications.

32. The computer readable medium of claim 30 , wherein at least one entry of the lookup table is provided in signaling by a video encoder that generated the coded video data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 2, 2022
From: DU, YIXIN; TOURAPIS, ALEXANDROS; ZHENG, YUNFEI; XIN, JUN; NALCI, ALICAN; GUO, MEI; WU, YEQING; WU, HSI-JUNG
To: APPLE INC.
Reel/Frame 061631/0422 →
Continuity (2)
Provisional Application 63277326 · Nov 9, 2021
Related Publication 20230143147A1 · May 11, 2023
References Cited (14)
US 20170019685A1 · Morigami · 2017 [cited by examiner]
US 20180352225A1 · Guo · 2018 [cited by examiner]
US 20200029095A1 · Karczewicz et al. · 2020 [cited by applicant]
US 20230048262A1 · Sim · 2023 [cited by examiner]
US 20240155119A1 · Strom et al. · 2024 [cited by examiner]
Webpage;“Git at Googe”; https://aomedia.googlesource.com/aom/+/refs/tags/research-v2.0.0; Oct. 27, 2022; 14 pages. [cited by applicant]
S. Midtskogen et al.; The AV1 Constrained Directional Enhancement Filter (CDEF); Oct. 28, 2017; 5 pages. [cited by applicant]
Y. Du et al.; “Cross-Component Sample Offset”; Alliance for Open Media, Codec Working Group; Document: CWG-B022-v3; Mar. 17, 2021; 6 pages. [cited by applicant]
X. Zhao et al.; “AOM Common Test Conditions v2.0”; Alliance for Open Media, Codec Working Group; Document: CWG-B075o; Aug. 31, 2021; 28 pages. [cited by applicant]
C. Rosewarne; “High Efficiency Video Calling (HEVC) Test Model 16 (HM 16)”; Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 40th Meeting: by teleconference; Jun. 24-Jul. 1,… [cited by applicant]
A. M. Tourapis et al.: “Clipping During 4:2:0 to 4:4:4 Conversions”; Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11; 21st Meeting: Warsaw, PL, Jun. 19-26, 2015; Docum… [cited by applicant]
J. Chen et al.; “Algorithm Description for Versatile Video Coding and Test Model 13 (VTM 13)”; Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29; 22nd Meeting by teleconference; Apr. 20-28, 202… [cited by applicant]
European Patent Application No. 22205086.6; Extended Search Report; dated Mar. 15, 2023; 10 pages. [cited by applicant]
Kuo et al.; “AHG12: Cross-component Sample Adaptive Offset”; Document: JVET-V0153-v2; Joint Video Experts Team; 20-28, Apr. 2021; 4 pages. [cited by applicant]