IP Library Granted Patent US 12,238,337
Granted Patent B2
US 12,238,337 · App. 17/549,631 · Granted Feb 25, 2025

Methods and system of subblock transform for video coding

Inventors: Xiaoyu Xiu (Beijing, CN); Yi-Wen Chen (Beijing, CN); Xianglin Wang (Beijing, CN); Shuiming Ye (Beijing, CN); Tsung-Chuan Ma (Beijing, CN); Hong-Jheng Jhu (Beijing, CN)
Assignee: BEIJING DAJIA INTERNET INFORMATION TECHNOLOGY CO., LTD.
H04N19/70H04N19/18H04N19/186H04N19/625H04N19/96
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,238,337
App. No.
17/549,631
Granted
Feb 25, 2025
Kind
B2
Abstract

A computing device performs a method of decoding video data by receiving, from a bitstream, a first syntax element associated with an SBT coding unit that indicates that there exists at least one non-zero transform coefficient in the SBT coding unit; determining a first transform unit that includes non-zero transform coefficients; receiving a second syntax element associated with a first chroma component of the first transform unit, a third syntax element associated with a second chroma component, and a fourth syntax element associated with luma component of the first transform unit; decoding the transform coefficients of chroma and luma components of the first transform unit, based on the second, third and fourth syntax elements; and setting transform coefficients of luma and chroma components of rest of the plurality of transform units to zeros.

Claims (49)

1. A method of video decoding, comprising:

receiving, from a bitstream, a first syntax element associated with a coding unit, wherein the first syntax element indicates whether there exists at least one non-zero transform coefficient in the coding unit;

in response to receiving the first syntax element indicating that there exists at least one non-zero transform coefficient in the coding unit, receiving a second syntax element indicating whether the coding unit is coded with a sub-block transform (SBT) mode;

in response to receiving the second syntax element indicating that the coding unit is coded with the SBT mode,

for a first transform unit of a plurality of transform units partitioned from the coding unit, receiving, from the bitstream, a third syntax element associated with a first chroma component of the first transform unit to indicate whether all transform coefficients of the first chroma component are zeros, receiving a fourth syntax element associated with a second chroma component of the first transform unit to indicate whether all transform coefficients of the second chroma component are zeros, and receiving a fifth syntax element associated with luma component of the first transform unit to indicate whether all transform coefficients of the luma component are zeros only when at least one of the third and fourth syntax elements is equal to one, wherein the first transform unit includes non-zero transform coefficients;

decoding the transform coefficients of chroma and luma components of the first transform unit, based on the third, fourth and fifth syntax elements; and

setting transform coefficients of luma and chroma components of rest of the plurality of transform units to zeros,

wherein the third syntax element is equal to 1 and the fourth syntax element is equal to 0 or 1, or the third syntax element is equal to 0 and the fourth syntax element is equal to 1.

2. The method of claim 1 , wherein the first transform unit is the only one of the plurality of transform units that includes non-zero transform coefficients.

3. The method of claim 1 , further comprising:

setting the fifth syntax element to one when both the third and fourth syntax elements are zeros.

4. The method of claim 1 , wherein the coding unit is partitioned into a ternary tree partition scheme that divides a coding unit into three transform units, and wherein the transform coefficients of two of the three transform units are set to zeros.

5. The method of claim 4 , wherein the three transform units are vertically adjacent to each other and include a top transform unit, a center transform unit, and a bottom transform unit, and wherein the top transform unit and the bottom transform unit have all-zero coefficients.

6. The method of claim 5 , wherein DST-VII transform is applied in a horizontal direction, and DCT-II transform is applied in a vertical direction of the center transform unit.

7. The method of claim 4 , wherein the three transform units are horizontally adjacent to each other and include a leftmost transform unit, a center transform unit, and a right rightmost transform unit, and the leftmost transform unit and the rightmost transform unit have all-zero transform coefficients.

8. The method of claim 7 , wherein DST-VII transform is applied in a vertical direction, and DCT-II transform is applied in a horizontal direction of the center transform unit.

9. A computing device comprising:

one or more processors;

memory coupled to the one or more processors; and

a plurality of programs stored in the memory that, when executed by the one or more processors, cause the computing device to perform acts comprising:

receiving, from a bitstream, a first syntax element associated with a coding unit, wherein the first syntax element indicates whether there exists at least one non-zero transform coefficient in the coding unit;

in response to receiving the first syntax element indicating that there exists at least one non-zero transform coefficient in the coding unit, receiving a second syntax element indicating whether the coding unit is coded with a sub-block transform (SBT) mode;

in response to receiving the second syntax element indicating that the coding unit is coded with the SBT mode,

for a first transform unit of a plurality of transform units partitioned from the coding unit,

receiving, from the bitstream, a third syntax element associated with a first chroma component of the first transform unit to indicate whether all transform coefficients of the first chroma component are zeros, receiving a fourth syntax element associated with a second chroma component of the first transform unit to indicate whether all transform coefficients of the second chroma component are zeros, and receiving a fifth syntax element associated with luma component of the first transform unit to indicate whether all transform coefficients of the luma component are zeros only when at least one of the third and fourth syntax elements is equal to one, wherein the first transform unit includes non-zero transform coefficients;

decoding the transform coefficients of chroma and luma components of the first transform unit, based on the third, fourth and fifth syntax elements; and

setting transform coefficients of luma and chroma components of rest of the plurality of transform units to zeros,

wherein the third syntax element is equal to 1 and the fourth syntax element is equal to 0 or 1, or the third syntax element is equal to 0 and the fourth syntax element is equal to 1.

10. The computing device of claim 9 , wherein the first transform unit is the only one of the plurality of transform units that includes non-zero transform coefficients.

11. The computing device of claim 9 , wherein the acts further comprise:

setting the fifth syntax element to one when both the third and fourth syntax elements are zeros.

12. The computing device of claim 9 , wherein the coding unit is partitioned into a ternary tree partition scheme that divides a coding unit into three transform units, and wherein the transform coefficients of two of the three transform units are set to zeros.

13. The computing device of claim 12 , wherein the three transform units are vertically adjacent to each other and include a top transform unit, a center transform unit, and a bottom transform unit, and wherein the top transform unit and the bottom transform unit have all-zero coefficients.

14. The computing device of claim 13 , wherein DST-VII transform is applied in a horizontal direction, and DCT-11 transform is applied in a vertical direction of the center transform unit.

15. The computing device of claim 12 , wherein the three transform units are horizontally adjacent to each other and include a leftmost transform unit, a center transform unit, and a right rightmost transform unit, and the leftmost transform unit and the rightmost transform unit have all-zero transform coefficients.

16. The computing device of claim 15 , wherein DST-VII transform is applied in a vertical direction, and DCT-11 transform is applied in a horizontal direction of the center transform unit.

17. A method for storing a bitstream, comprising:

performing an encoding method to generate a bitstream; and

storing the bitstream on a non-transitory computer readable storage medium,

wherein the encoding method comprises:

determining a first syntax element associated with a coding unit, wherein the first syntax element indicates whether there exists at least one non-zero transform coefficient in the coding unit;

in response to the first syntax element indicating that there exists at least one non-zero transform coefficient in the coding unit, determining a second syntax element indicating whether the coding unit is coded with a sub-block transform (SBT) mode;

in response to the second syntax element indicating that the coding unit is coded with the SBT mode,

for a first transform unit of a plurality of transform units partitioned from the coding unit,

determining a third syntax element associated with a first chroma component of the first transform unit to indicate whether all transform coefficients of the first chroma component are zeros, determining a fourth syntax element associated with a second chroma component of the first transform unit to indicate whether all transform coefficients of the second chroma component are zeros, and determining a fifth syntax element associated with luma component of the first transform unit to indicate whether all transform coefficients of the luma component are zeros only when at least one of the third and fourth syntax elements is equal to one, wherein the first transform unit includes non-zero transform coefficients;

decoding the transform coefficients of chroma and luma components of the first transform unit, based on the third, fourth and fifth syntax elements; and

setting transform coefficients of luma and chroma components of rest of the plurality of transform units to zeros,

wherein the third syntax element is equal to 1 and the fourth syntax element is equal to 0 or 1, or the third syntax element is equal to 0 and the fourth syntax element is equal to 1.

18. The method of claim 17 , wherein the first transform unit is the only one of the plurality of transform units that includes non-zero transform coefficients.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 27, 2022
From: XIU, XIAOYU; CHEN, YI-WEN; WANG, XIANGLIN; YE, SHUIMING; MA, TSUNG-CHUAN; JHU, HONG-JHENG
To: BEIJING DAJIA INTERNET INFORMATION TECHNOLOGY CO., LTD.
Reel/Frame 059743/0460 →
Continuity (3)
Continuation PCTUS2020037445 · Jun 12, 2020
Provisional Application 62861230 · Jun 13, 2019
Related Publication 20220109886A1 · Apr 7, 2022
References Cited (32)
US 10313688B2 · Terada et al. · 2019 [cited by applicant]
US 20120189052A1 · Karczewicz et al. · 2012 [cited by applicant]
US 20120308148A1 · Kim · 2012 [cited by examiner]
US 20130034152A1 · Song · 2013 [cited by examiner]
US 20140078249A1 · Wang · 2014 [cited by applicant]
US 20140092966A1 · Kazui et al. · 2014 [cited by applicant]
US 20160156917A1 · Ugur et al. · 2016 [cited by applicant]
US 20160219276A1 · Li · 2016 [cited by examiner]
US 20180288437A1 · Hsiang · 2018 [cited by examiner]
US 20190149822A1 · Kim et al. · 2019 [cited by applicant]
US 20220094939A1 · Chuang · 2022 [cited by examiner]
CN 103329529A · 2013 [cited by applicant]
CN 103716621A · 2014 [cited by applicant]
CN 117478913A · 2024 [cited by applicant]
Bross et al: “Versatile Video Coding (Draft 5}”,—14. JVET Meeting; Mar. 19, 2019-Mar. 27, 2019; Geneva; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/;WG11 and ITU-T SG. 16},, No. JVET-N1001-v8; JVET-N1001 Jun.… [cited by examiner]
Zhao et al. “CEG: ROT-like sub-block transform for inter blocks (Test 6.4.4)”, 13. JVET Meeting; Jan. 9, 2019-Jan. 18, 2019: Marrakech: (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG. 16 },, No… [cited by examiner]
Beijing Dajia Internet Information Technology Co., Ltd., CN202210245849.X, First Office Action, Jan. 11, 2023, 23 pgs. [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 5)”, Document: JVET-N1001-v7v8, Joint Video Experts Team (JVET), of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 400 pgs. [cited by applicant]
Liang et al., “The Real-time Arithmetic and Realization of Encoding and Decoding of DGPS's RTCM Data”, Global Position System (GPS), Differential GPS(DGPS), RTCM, MSCOMM, Visual C++, TP311; TP391, 1002-8331-(2006) 11-02… [cited by applicant]
Huawei Technologies Co., Ltd., “CE6: RQT-like sub-block transform for inter blocks (Test 6.4.4)”, Document: JVET-M0141-v21, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting… [cited by applicant]
Beijing Dajia Internet Information Technology Co. Ltd. et al., International Search Report and Written Opinion, PCT/US2020/037445, Sep. 22, 2020, 8 pgs. [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding (Draft 5),” JVET-N1001-v8, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1S/C 29/WG 11 14th Meeting: Geneva, CH, Jun. 11, 2019, pp. 1-385, pp. 53-62, … [cited by applicant]
Yin Zhao et al., “CE6: RQT-like sub-block transform for inter blocks (Test 6.4.4),” JVET-M0141-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 WG 11, 13th Meeting: Marrakech, MA, Jan. 11,… [cited by applicant]
Extended Search Report for EP 20823103.5 mailed May 22, 2023, 14 pages. [cited by applicant]
Bross, Benjamin , et al., “Versatile Video Coding (Draft 5)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 423 pages. [cited by applicant]
Chen, Jianle , et al., “Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27… [cited by applicant]
Chernyak, Roman , et al., “CBF flags signalling in VVC”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 7 pages. [cited by applicant]
Jung, Jason , et al., “Non-CE6: Modifications on sub-block transform”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 3 pages. [cited by applicant]
Zhao, Yin , et al., “CE6: RQT-like sub-block transform for inter blocks (Test 6.4.4)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, 4 p… [cited by applicant]
Zhao, Yin , et al., “CE6: Sub-block transform for inter blocks (CE6.4.1)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, 18 pages. [cited by applicant]
Chinese Office Action corresponding to Chinese Patent Application No. 202311490927.3, (Foreign Text, 3 Pages, English Translation Thereof, 3 Pages) (Oct. 12, 2024),. [cited by applicant]
Notice of Allowance corresponding to Chinese Patent Application No. 202311490927.3, (Foreign Text, 2 Pages, English Translation Thereof, 2 Pages) (Jan. 7, 2025),. [cited by applicant]