Granted Patent B2

US 12,323,625 · App. 17/411,170 · Granted Jun 3, 2025

Inventors: Li Zhang (San Diego, CA); Kai Zhang (San Diego, CA); Hongbin Liu (Beijing, CN); Yue Wang (Beijing, CN)

Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.

H04N19/70H04N19/11H04N19/12H04N19/124H04N19/132H04N19/159H04N19/176H04N19/186H04N19/46H04N19/593H04N19/61

Abstract

Claims (71)

1. A method of processing video data, comprising:

performing a conversion between a current video block of a video and a bitstream of the video,

wherein the conversion comprises:

determining whether a first index syntax element is included in the bitstream based on a flag in sequence parameter set (SPS) and a position of a last non-zero coefficient in a residual of the current video block;

determining whether a second index syntax element is included in the bitstream based on a value of the first index syntax element, wherein the second index syntax element is used to determine a transform matrix of a primary transform to be used on the current video block from multiple transform sets;

determining whether to apply a secondary transform tool on the current video block based on the value of the first index syntax element;

determining, in response to the secondary transform tool being applied on the current video block, a transform matrix set used in the secondary transform tool based on whether a matrix-based intra prediction mode is applied to the current video block; and

determining a transform kernel to be used for the secondary transform tool based on the value of the first index syntax element;

wherein the first index syntax element is included in the bitstream in a case that the flag in SPS is equal to one and the last non-zero coefficient is located in the residual of a coding group of the current video block,

wherein the second index syntax element is not included in the bitstream in a case that the secondary transform tool is applied on the current video block,

wherein the matrix-based intra prediction mode includes determining prediction values of the current video block based on a matrix vector multiplication; and

wherein, using the secondary transform tool:

during encoding, a forward secondary transform is applied to an output of a forward primary transform applied to a residual of the current video block prior to quantization, or

during decoding, an inverse secondary transform is applied to an output of dequantization of the current video block before applying an inverse primary transform; and

wherein a context increasement set corresponding to a second bin of the first index syntax element includes only one context increasement, and the only one context increasement is 2.

2. The method of claim 1 , wherein determining prediction values of the current video block is further based on a boundary down-sampling operation on reference samples based on a size of the current video block and selectively followed by an up-sampling operation.

3. The method of claim 1 , wherein the secondary transform tool includes a low frequency non-separable transform tool.

4. The method of claim 1 , wherein the conversion includes encoding the current video block into the bitstream.

5. The method of claim 1 , wherein the conversion includes decoding the current video block from the bitstream.

6. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

perform a conversion between a current video block of a video and a bitstream of the video, wherein the conversion comprises:

determining whether a first index syntax element is included in the bitstream based on a flag in sequence parameter set (SPS) and a position of a last non-zero coefficient in a residual of the current video block;

determining whether a second index syntax element is included in the bitstream based on a value of the first index syntax element, wherein the second index syntax element is used to determine a transform matrix of a primary transform to be used on the current video block from multiple transform sets;

determining whether to apply a secondary transform tool on the current video block based on the value of the first index syntax element;

determining, in response to the secondary transform tool being applied on the current video block, a transform matrix set used in the secondary transform tool based on whether a matrix-based intra prediction mode is applied to the current video block; and

determining a transform kernel to be used for the secondary transform tool based on the value of the first index syntax element;

wherein the first index syntax element is included in the bitstream in a case that the flag in SPS is equal to one and the last non-zero coefficient is located in the residual of a coding group of the current video block,

wherein the second index syntax element is not included in the bitstream in a case that the secondary transform tool is applied on the current video block,

wherein the matrix-based intra prediction mode includes determining prediction values of the current video block based on a matrix vector multiplication; and

wherein, using the secondary transform tool:

during encoding, a forward secondary transform is applied to an output of a forward primary transform applied to a residual of the current video block prior to quantization, or

during decoding, an inverse secondary transform is applied to an output of dequantization of the current video block before applying an inverse primary transform; and

wherein a context increasement set corresponding to a second bin of the first index syntax element includes only one context increasement, and the only one context increasement is 2.

7. The apparatus of claim 6 , wherein determining prediction values of the current video block is further based on a boundary down-sampling operation on reference samples based on a size of the current video block and selectively followed by an up-sampling operation.

8. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

perform a conversion between a current video block of a video and a bitstream of the video, wherein the conversion comprises:

determining whether a first index syntax element is included in the bitstream based on a flag in sequence parameter set (SPS) and a position of a last non-zero coefficient in a residual of the current video block;

determining whether a second index syntax element is included in the bitstream based on a value of the first index syntax element, wherein the second index syntax element is used to determine a transform matrix of a primary transform to be used on the current video block from multiple transform sets;

determining whether to apply a secondary transform tool on the current video block based on the value of the first index syntax element;

determining, in response to the secondary transform tool being applied on the current video block, a transform matrix set used in the secondary transform tool based on whether a matrix-based intra prediction mode is applied to the current video block; and

determining a transform kernel to be used for the secondary transform tool based on the value of the first index syntax element;

wherein the first index syntax element is included in the bitstream in a case that the flag in SPS is equal to one and the last non-zero coefficient is located in the residual of a coding group of the current video block,

wherein the second index syntax element is not included in the bitstream in a case that the secondary transform tool is applied on the current video block,

wherein the matrix-based intra prediction mode includes determining prediction values of the current video block based on a matrix vector multiplication; and

wherein, using the secondary transform tool:

during encoding, a forward secondary transform is applied to an output of a forward primary transform applied to a residual of the current video block prior to quantization, or

during decoding, an inverse secondary transform is applied to an output of dequantization of the current video block before applying an inverse primary transform; and

wherein a context increasement set corresponding to a second bin of the first index syntax element includes only one context increasement, and the only one context increasement is 2.

9. A method for storing a bitstream of a video, comprising:

generating the bitstream for a current video block of the video; and

storing the bitstream in a non-transitory computer-readable recording medium;

wherein the generating comprises:

determining whether a first index syntax element is included in the bitstream based on a flag in sequence parameter set (SPS) and a position of a last non-zero coefficient in a residual of the current video block;

determining whether a second index syntax element is included in the bitstream based on a value of the first index syntax element, wherein the second index syntax element is used to determine a transform matrix of a primary transform to be used on the current video block from multiple transform set;

determining whether to apply a secondary transform tool on the current video block based on the value of the first index syntax element;

determining, in response to the secondary transform tool being applied on the current video block, a transform matrix set used in the secondary transform tool based on whether a matrix-based intra prediction mode is applied to the current video block; and

determining a transform kernel to be used for the secondary transform tool based on the value of the first index syntax element;

wherein the first index syntax element is included in the bitstream in a case that the flag in SPS is equal to one and the last non-zero coefficient is located in the residual of a coding group of the current video block,

wherein the second index syntax element is not included in the bitstream in a case that the secondary transform tool is applied on the current video block,

wherein the matrix-based intra prediction mode includes determining prediction values of the current video block based on a matrix vector multiplication; and

wherein, using the secondary transform tool:

during encoding, a forward secondary transform is applied to an output of a forward primary transform applied to a residual of the current video block prior to quantization, or

during decoding, an inverse secondary transform is applied to an output of dequantization of the current video block before applying an inverse primary transform; and

wherein a context increasement set corresponding to a second bin of the first index syntax element includes only one context increasement, and the only one context increasement is 2.

10. The apparatus of claim 6 , wherein the secondary transform tool includes a low frequency non-separable transform tool.

11. The non-transitory computer-readable storage medium of claim 8 , wherein determining prediction values of the current video block is further based on a boundary down-sampling operation on reference samples based on a size of the current video block and selectively followed by an up-sampling operation.

12. The non-transitory computer-readable storage medium of claim 8 , wherein the secondary transform tool includes a low frequency non-separable transform tool.

13. The method of claim 9 , wherein determining prediction values of the current video block is further based on a boundary down-sampling operation on reference samples based on a size of the current video block and selectively followed by an up-sampling operation.

14. The method of claim 9 , wherein the secondary transform tool includes a low frequency non-separable transform tool.

15. The method of claim 1 , wherein whether the first index syntax element being included in the bitstream is further based on a color component of the current video block.

16. The apparatus of claim 6 , wherein whether the first index syntax element being included in the bitstream is further based on a color component of the current video block.

Assignments (2)

Continuity (2)

References Cited (199)