IP Library › Granted Patent US 12,225,207
Granted Patent B2
US 12,225,207 · App. 18/207,578 · Granted Feb 11, 2025

Subblock intra and inter coding

Inventors: Xin Zhao (Santa Clara, CA); Guichun Li (San Jose, CA); Lien-Fei Chen (Hsinchu, TW); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/159H04N19/103H04N19/129H04N19/172H04N19/176H04N19/186H04N19/52H04N19/593H04N19/60H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,225,207
App. No.
18/207,578
Granted
Feb 11, 2025
Kind
B2
Abstract

Processing circuitry receives a coded bitstream carrying at least a picture, determines that a current coding unit (CU) in the picture is coded in a subblock based inter prediction mode based on a first syntax element value in the coded bitstream, and determines that one or more first subblocks in the current CU that is coded in the subblock based inter prediction mode are coded by intra prediction. The processing circuitry reconstructs one or more second subblocks of the current CU by inter prediction based on the subblock based inter prediction mode, the one or more second subblocks do not overlap with the one or more first subblocks in the current CU. The processing circuitry reconstructs the one or more first subblocks of the current CU by the intra prediction while the current CU is coded in the subblock based inter prediction mode.

Claims (52)

1. A method of video decoding, comprising:

receiving a coded bitstream carrying at least a picture that includes a block comprising a plurality of subblocks;

determining that a current coding unit (CU) in the picture is coded in a subblock based inter prediction mode based on a first syntax element value in the coded bitstream;

determining that one or more first subblocks in the current CU that is coded in the subblock based inter prediction mode are coded by intra prediction;

reconstructing one or more second subblocks of the current CU by inter prediction based on the subblock based inter prediction mode, wherein the one or more second subblocks do not overlap with the one or more first subblocks in the current CU; and

reconstructing the one or more first subblocks of the current CU by the intra prediction while the current CU is coded in the subblock based inter prediction mode.

2. The method of claim 1 , wherein the subblock based inter prediction mode comprises at least one of an affine mode, a regression based inter prediction mode or a subblock-based temporal motion vector prediction (SbTMVP) mode.

3. The method of claim 1 , wherein the determining that the one or more first subblocks in the current CU are coded by the intra prediction comprises:

determining a first flag that indicates whether at least one subblock in the current CU is coded by the intra prediction; and

decoding, from the bitstream, second flags respectively associated with subblocks in the current CU, a second flag associated with a subblock in the current CU indicating whether the subblock being coded by the intra prediction.

4. The method of claim 3 , wherein the determining the first flag further comprises:

decoding the first flag from the bitstream.

5. The method of claim 3 , wherein the determining the first flag further comprises:

deriving that the first flag is of a true value in response to a determination that a collocated block for a subblock in the current CU has no valid motion vector when the current CU is coded in a subblock-based temporal motion vector prediction (SbTMVP) mode; or

deriving that the first flag is of a false value in response to a determination that each subblock in the current CU has a valid motion vector when the current CU is coded in the SbTMVP mode.

6. The method of claim 1 , wherein the one or more second subblocks of the current CU are reconstructed before the one or more first subblocks of the current CU, and the one or more first subblocks of the current CU are reconstructed following a specific order that is determined based on positions of the one or more second subblocks.

7. The method of claim 1 , further comprising:

reconstructing the one or more second subblocks of the current CU according to a raster scanning order; and

reconstructing the one or more first subblocks of the current CU according to the raster scanning order after the one or more second subblocks being reconstructed.

8. The method of claim 6 , further comprising:

for a first subblock in the one or more first subblocks, reconstructing the first subblock according to neighboring inter coded subblocks and neighboring intra coded subblocks in the current CU.

9. The method of claim 1 , further comprising:

determining a transform unit size to be a size of subblocks in response to a first flag being true, the first flag indicating whether at least one subblock in the current CU is coded by the intra prediction; and

determining the transform unit size to be a size of the current CU in response to the first flag being false.

10. The method of claim 1 , further comprising:

performing an inverse transform to obtain a transform unit of a size of the current CU; and

determining residual values corresponding to the one or more second subblocks according to the transform unit.

11. The method of claim 1 , further comprising:

performing inverse transforms to obtain one or more transform units of a size of a subblock in the current CU, the one or more transform units respectively corresponding to the one or more first subblocks; and

determining residual values corresponding to the one or more first subblocks according to the one or more transform units.

12. The method of claim 1 , wherein the reconstructing the one or more first subblocks further comprises:

reconstructing a first subblock in the one or more first subblocks based on an intra prediction mode that is selected from at least one of a set of most portable modes, as a set of selected smooth modes or a set of non-directional modes.

13. The method of claim 1 , wherein the reconstructing the one or more first subblocks further comprises:

reconstructing a first subblock in the one or more first subblocks according to at least one of one or more right columns, one or more bottom rows, one or more corner samples, one or more above right rows, or one or more bottom left columns.

14. The method of claim 1 , wherein the reconstructing the one or more first subblocks further comprises:

reconstructing a luma component and at least a chroma component of a subblock in the one or more first subblocks by the intra prediction.

15. The method of claim 1 , wherein the reconstructing the one or more first subblocks further comprises:

in response to the one or more first subblocks in the current CU are coded by the intra prediction, determining the inter prediction with constraints for reconstructing the one or more second subblocks.

16. The method of claim 15 , wherein the determining the inter prediction further comprises:

determining to use uni-prediction for the inter prediction.

17. The method of claim 1 , wherein the reconstructing the one or more first subblocks further comprises:

determining a predictor of an intra prediction mode from a neighboring pixel of the current CU, the intra prediction mode including at least one of a most probable mode (MPM), a decoder-side intra mode derivation (DIMD) mode, a template based intra mode derivation (TIMD) mode, or a directional intra prediction mode.

18. An apparatus of video processing, comprising processing circuitry configured to:

receive a coded bitstream carrying at least a picture that includes a block comprising a plurality of subblocks;

determine that a current coding unit (CU) in the picture is coded in a subblock based inter prediction mode based on a first syntax element value in the coded bitstream;

determine that one or more first subblocks in the current CU that is coded in the subblock based inter prediction mode are coded by intra prediction;

reconstruct one or more second subblocks of the current CU by inter prediction based on the subblock based inter prediction mode, wherein the one or more second subblocks do not overlap with the one or more first subblocks in the current CU; and

reconstruct the one or more first subblocks of the current CU by the intra prediction while the current CU is coded in the subblock based inter prediction mode.

19. The apparatus of claim 18 , wherein the subblock based inter prediction mode comprises at least one of an affine mode, a regression based inter prediction mode or a subblock-based temporal motion vector prediction (SbTMVP) mode.

20. The apparatus of claim 18 , wherein the processing circuitry is further configured to:

determine a first flag that indicates whether at least one subblock in the current CU is coded by the intra prediction; and

decode, from the bitstream, second flags respectively associated with subblocks in the current CU, a second flag associated with a subblock in the current CU indicating whether the subblock being coded by the intra prediction.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2023
From: ZHAO, XIN; LI, GUICHUN; CHEN, LIEN-FEI; LIU, SHAN
To: TENCENT AMERICA LLC
Reel/Frame 063900/0125 →
Continuity (2)
Provisional Application 63359155 · Jul 7, 2022
Related Publication 20240015305A1 · Jan 11, 2024
References Cited (7)
US 9420293B2 · Song · 2016 [cited by examiner]
US 11706437B2 · Misra · 2023 [cited by examiner]
US 11831816B2 · Wang · 2023 [cited by examiner]
High Efficiency Video Coding, Rec. ITU-T H.265 v4 Dec. 2016, pp. 1-664. [cited by applicant]
ITU-T and ISO/IEC, “Versatile Video Coding”, ITU-T Rec. H.266 and ISO/IEC 23090-3, 2020, pp. 1-516. [cited by applicant]
H. Yang, H. Chen, J. Chen, S. Esenlik, S. Sethuraman, X. Xiu, E. Alshina, and J. Luo, “Subblock-Based Motion Derivation and Inter Prediction Refinement in Versatile Video Coding Standard”, IEEE Trans. Circuits Syst. Vid… [cited by applicant]
M. Coban, et al., “Algorithm description of Enhanced Compression Model 4 (ECM 4)”, JVET-Y2025-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 23rd Meeting, by teleconference, Jul. 7-16, … [cited by applicant]