IP Library Granted Patent US 12,075,030
Granted Patent B2
US 12,075,030 · App. 18/081,404 · Granted Aug 27, 2024

Subpicture dependent signaling in video bitstreams

Inventors: Kai Zhang (San Diego, CA); Zhipin Deng (Beijing, CN); Hongbin Liu (Beijing, CN); Li Zhang (San Diego, CA); Jizheng Xu (San Diego, CA)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/105H04N19/103H04N19/119H04N19/132H04N19/157H04N19/167H04N19/172H04N19/176H04N19/186H04N19/1883H04N19/30H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,075,030
App. No.
18/081,404
Granted
Aug 27, 2024
Kind
B2
Abstract

A method of video processing includes performing a conversion between a video unit of a video and a coded representation of the video using at least a video picture. Only one of a subpicture coding mode or a resolution-changing coding mode is enabled for the video unit. The subpicture coding mode is a mode in which the video picture is divided into multiple subpictures, and the resolution-changing coding mode is a mode in which a resolution of the video picture is adjusted during the conversion.

Claims (69)

1. A method of processing video data, comprising:

determining, for a first conversion between a first video block of a first picture of a video and a bitstream of the video, that an affine motion mode is applied for the first video block;

constructing, for the first video block, a motion candidate list comprising at least one constructed motion candidate that is generated based on a temporal motion information; and

deriving prediction samples based on the motion candidate list and performing the first conversion,

wherein a position from which the temporal motion information for generating the at least one constructed motion candidate is derived is constrained to be within a first subpicture of the first picture,

wherein the temporal motion information is unavailable in response to the position being not located within the first subpicture, and wherein the position is at a bottom-right corner of the first video block, and

wherein a subpicture coding mode in which a video picture is divided into multiple subpictures is enabled for the first picture.

2. The method of claim 1 , wherein the first subpicture is treated as a picture in the first conversion excluding an in-loop filtering operation.

3. The method of claim 1 , wherein deriving the prediction samples comprises:

applying an integer sample fetching process to generate fetched samples, wherein in the integer sample fetching process, a position at which an integer sample is fetched in a reference picture of the first video block is constrained in a second subpicture of the reference picture;

generating gradients based on the fetched samples; and

applying a prediction refinement with an optical flow process based on the gradients to derive the prediction samples.

4. The method of claim 1 , further comprising:

determining, for a second conversion between a video unit of the video and the bitstream of the video, that the subpicture coding mode is enabled for the video unit, wherein the video unit comprises at least the first picture;

partitioning a video picture in the video unit into multiple coding tree blocks, and partitioning the video picture into one or more subpictures according to a rule; and

performing the second conversion between the video unit and the bitstream based on the partitioning,

wherein the rule specifies that a boundary between any two subpictures is also a boundary between two coding tree blocks, and that a coding tree block is not allowed to cover more than one subpicture.

5. The method of claim 4 , wherein a first syntax element and a second syntax element are included in the bitstream, and

wherein the first syntax element specifies a width of a subpicture in units of a width of a coding tree block, and the second syntax element specifies a height of the subpicture in units of a height of the coding tree block.

6. The method of claim 4 , wherein any position in the video picture is constrained to be covered by only one subpicture.

7. The method of claim 4 , wherein a third syntax element indicating the subpicture coding mode is enabled is included in a sequence parameter set in the bitstream.

8. The method of claim 1 , further comprising:

determining, for a third conversion between a video unit of the video and the bitstream of the video, whether a resolution-changing coding mode is enabled for the video unit, wherein the video unit comprises at least the first picture; and

determining, in a case that the resolution-changing coding mode is enabled for the video unit, the subpicture coding mode is disabled for the video unit,

wherein the resolution-changing coding mode is a mode in which a spatial resolution is allowed to change between video pictures of the video unit.

9. The method of claim 8 , wherein in a case that the resolution-changing coding mode is disabled, a width of the video picture in the video unit is set to be equal to a maximum width allowed for the video pictures in the video unit.

10. The method of claim 9 , wherein a fourth syntax element is included in the bitstream to indicate the maximum width allowed for the video pictures in the video unit, and wherein a value of the fourth syntax element is restricted to be no smaller than 8.

11. The method of claim 10 , wherein a fifth syntax element is included in the bitstream to indicate a maximum height allowed for the video pictures in the video unit, and a value of the fifth syntax element is restricted to be no smaller than 8.

12. The method of claim 1 , further comprising:

determining, for the first conversion between the first video block of the first picture of the video and the bitstream of the video, that a first prediction mode is applied on the first video block;

maintaining, for the first prediction mode, a virtual buffer comprising reference samples derived from blocks of sample values of the first picture;

deriving, for the first video block, prediction samples based on the determining; and

performing the first conversion between the first video block and the bitstream,

wherein in the first prediction mode, the reference samples determined by a block vector in the virtual buffer are used for the derivation of the prediction samples, and

wherein the subpicture coding mode is enabled for the first picture, and the virtual buffer is reset before processing a subpicture of the first picture.

13. The method of claim 12 , wherein the reference samples in the virtual buffer are reset to be −1.

14. The method of claim 12 , further comprising:

determining, for a fourth conversion between a second video block of a second picture of the video and the bitstream, that a second prediction mode is applied on the second video block;

maintaining, for the second prediction mode, a predictor palette; and

performing the fourth conversion between the second video block and the bitstream based on the second prediction mode,

wherein in the second prediction mode, reconstructed samples are represented by a set of representative color values, and the set of representative color values comprises at least one of 1) palette predictors, 2) escaped samples, or 3) palette information included in the bitstream, and wherein the subpicture coding mode is enabled for the second picture, and entry list of the predictor palette is reset before processing a subpicture of the second picture.

15. The method of claim 1 , wherein the first conversion comprises encoding the video into the bitstream.

16. The method of claim 1 , wherein the first conversion comprises decoding the video from the bitstream.

17. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, for a first conversion between a first video block of a first picture of a video and a bitstream of the video, that an affine motion mode is applied for the first video block;

construct, for the first video block, a motion candidate list comprising at least one constructed motion candidate that is generated based on a temporal motion information; and

derive prediction samples based on the motion candidate list and performing the first conversion,

wherein a position from which the temporal motion information for generating the at least one constructed motion candidate is derived is constrained to be within a first subpicture of the first picture,

wherein the temporal motion information is unavailable in response to the position being not located within the first subpicture, and wherein the position is at a bottom-right corner of the first video block, and

wherein a subpicture coding mode in which a video picture is divided into multiple subpictures is enabled for the first picture.

18. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, for a first conversion between a first video block of a first picture of a video and a bitstream of the video, that an affine motion mode is applied for the first video block;

construct, for the first video block, a motion candidate list comprising at least one constructed motion candidate that is generated based on a temporal motion information; and

derive prediction samples based on the motion candidate list and performing the first conversion,

wherein a position from which the temporal motion information for generating the at least one constructed motion candidate is derived is constrained to be within a first subpicture of the first picture,

wherein the temporal motion information is unavailable in response to the position being not located within the first subpicture, and wherein the position is at a bottom-right corner of the first video block, and

wherein a subpicture coding mode in which a video picture is divided into multiple subpictures is enabled for the first picture.

19. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

determining, for generating a bitstream of a video from a first video block of a first picture of the video, that an affine motion mode is applied for the first video block;

constructing, for the first video block, a motion candidate list comprising at least one constructed motion candidate that is generated based on a temporal motion information; and

deriving prediction samples based on the motion candidate list and generating the bitstream,

wherein a position from which the temporal motion information for generating the at least one constructed motion candidate is derived is constrained to be within a first subpicture of the first picture,

wherein the temporal motion information is unavailable in response to the position being not located within the first subpicture, and wherein the position is at a bottom-right corner of the first video block, and

wherein a subpicture coding mode in which a video picture is divided into multiple subpictures is enabled for the first picture.

20. The apparatus of claim 17 , wherein the instructions, upon execution by the processor, further cause the processor to:

determine, for a second conversion between a video unit of the video and the bitstream of the video, that the subpicture coding mode is enabled for the video unit, wherein the video unit comprises at least the first picture;

partition a video picture in the video unit into multiple coding tree blocks, and partition the video picture into one or more subpictures according to a rule; and

perform the second conversion between the video unit and the bitstream based on the partitioning,

wherein the rule specifies that a boundary between any two subpictures is also a boundary between two coding tree blocks, and that a coding tree block is not allowed to cover more than one subpicture.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 31, 2023
From: DENG, ZHIPIN; LIU, HONGBIN
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 063816/0182 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 31, 2023
From: ZHANG, KAI; ZHANG, LI; XU, JIZHENG
To: BYTEDANCE INC.
Reel/Frame 063816/0283 →
Priority Claims (1)
WO PCT/CN2019/100114 · Aug 10, 2019 · international
Continuity (3)
Continuation 17665275 · Feb 4, 2022
Continuation PCTCN2020108175 · Aug 10, 2020
Related Publication 20230118260A1 · Apr 20, 2023
Cited By (6)
US 12,301,799 US 12,348,761 US 12,388,989 US 12,413,714 US 12,513,291 US 12,621,450