Luma based secondary transform matrix selection for video processing
A video processing method is provided to include: performing a conversion between a current video block of a video and a coded representation of the video, wherein the coded representation conforms to a format rule that specifies a last non-zero coefficient in a residual of the current video block and controls whether or how side information about a secondary transform tool is included in the coded representation, and wherein the secondary transform tool includes applying, during encoding, a forward secondary transform to an output of a forward primary transform applied to a residual of a video block prior to quantization, or applying, during decoding, an inverse secondary transform to an output of dequantization of the video block before applying an inverse primary transform.
1. A method of processing video data, comprising:
performing a conversion between a current video block of a video and a bitstream of the video,
wherein a location of a last non-zero coefficient in a residual of the current video block is determined based on at least one syntax element in the bitstream, and whether or how to include side information about a secondary transform tool in the bitstream is based on the location of the last non-zero coefficient,
wherein the secondary transform tool includes applying, during encoding, a forward secondary transform to an output of a forward primary transform applied to a residual of a video block prior to quantization, or applying, during decoding, an inverse secondary transform to an output of dequantization of the video block before applying an inverse primary transform,
wherein the side information comprises at least one of indication of applying the secondary transform tool or an index of transform kernels used in the secondary transform tool,
wherein the side information is excluded from the bitstream in a case that the last non-zero coefficient is not located in a region of the current video block to which the secondary transform tool is applied, and
wherein the side information is excluded from the bitstream in a case that the residual of the current video block includes no non-zero coefficient.
2. The method of claim 1 , wherein the secondary transform tool corresponds to a low frequency non-separable transform (LFNST) tool.
3. The method of claim 1 , wherein whether or how to include the side information about the secondary transform tool in the bitstream is not based on how many non-zero coefficients in the residual of the current video block.
4. The method of claim 1 , wherein the side information is included in the bitstream in a case that the last non-zero coefficient is located in a region of the current video block to which the secondary transform tool is applied.
5. The method of claim 1 , wherein in a case that a size of the current video block is 4*4 or 8*8, the region is corresponding to the first 8 coefficients in a top-left 4*4 coding group.
6. The method of claim 1 , wherein in a case that a height or width of the current video block are greater than or equal to 4, and a size of the current video block is not 4*4 and 8*8, the region is corresponding to a top-left 4*4 coding group.
7. The method of claim 1 , wherein the conversion includes encoding the current video block into the bitstream.
8. The method of claim 1 , wherein the conversion includes decoding the current video block from the bitstream.
9. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
perform a conversion between a current video block of a video and a bitstream of the video,
wherein a location of a last non-zero coefficient in a residual of the current video block is determined based on at least one syntax element in the bitstream, and whether or how to include side information about a secondary transform tool in the bitstream is based on the location of the last non-zero coefficient,
wherein the secondary transform tool includes applying, during encoding, a forward secondary transform to an output of a forward primary transform applied to a residual of a video block prior to quantization, or applying, during decoding, an inverse secondary transform to an output of dequantization of the video block before applying an inverse primary transform,
wherein the side information comprises at least one of indication of applying the secondary transform tool or an index of transform kernels used in the secondary transform tool,
wherein the side information is excluded from the bitstream in a case that the last non-zero coefficient is not located in a region of the current video block to which the secondary transform tool is applied, and
wherein the side information is excluded from the bitstream in a case that the residual of the current video block includes no non-zero coefficient.
10. The apparatus of claim 9 , wherein the secondary transform tool corresponds to a low frequency non-separable transform (LFNST) tool.
11. The apparatus of claim 9 , wherein whether or how to include the side information about the secondary transform tool in the bitstream is not based on how many non-zero coefficients in the residual of the current video block.
12. The apparatus of claim 9 , wherein in a case that a size of the current video block is 4*4 or 8*8, the region is corresponding to the first 8 coefficients in a top-left 4*4 coding group.
13. The apparatus of claim 9 , wherein in a case that a height or width of the current video block are greater than or equal to 4, and a size of the current video block is not 4*4 and 8*8, the region is corresponding to a top-left 4*4 coding group.
14. A non-transitory computer-readable storage medium storing instructions that cause a processor to:
perform a conversion between a current video block of a video and a bitstream of the video,
wherein a location of a last non-zero coefficient in a residual of the current video block is determined based on at least one syntax element in the bitstream, and whether or how to include side information about a secondary transform tool in the bitstream is based on the location of the last non-zero coefficient,
wherein the secondary transform tool includes applying, during encoding, a forward secondary transform to an output of a forward primary transform applied to a residual of a video block prior to quantization, or applying, during decoding, an inverse secondary transform to an output of dequantization of the video block before applying an inverse primary transform,
wherein the side information comprises at least one of indication of applying the secondary transform tool or an index of transform kernels used in the secondary transform tool,
wherein the side information is excluded from the bitstream in a case that the last non-zero coefficient is not located in a region of the current video block to which the secondary transform tool is applied, and
wherein the side information is excluded from the bitstream in a case that the residual of the current video block includes no non-zero coefficient.
15. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
generating the bitstream for a current video block of the video,
wherein a location of a last non-zero coefficient in a residual of the current video block is determined based on at least one syntax element in the bitstream, and whether or how to include side information about a secondary transform tool in the bitstream is based on the location of the last non-zero coefficient,
wherein the secondary transform tool includes applying, during encoding, a forward secondary transform to an output of a forward primary transform applied to a residual of a video block prior to quantization, or applying, during decoding, an inverse secondary transform to an output of dequantization of the video block before applying an inverse primary transform,
wherein the side information comprises at least one of indication of applying the secondary transform tool or an index of transform kernels used in the secondary transform tool,
wherein the side information is excluded from the bitstream in a case that the last non-zero coefficient is not located in a region of the current video block to which the secondary transform tool is applied, and
wherein the side information is excluded from the bitstream in a case that the residual of the current video block includes no non-zero coefficient.
16. The non-transitory computer-readable storage medium of claim 14 , wherein the secondary transform tool corresponds to a low frequency non-separable transform (LFNST) tool.
17. The non-transitory computer-readable storage medium of claim 14 , wherein whether or how to include the side information about the secondary transform tool in the bitstream is not based on how many non-zero coefficients in the residual of the current video block.
18. The non-transitory computer-readable storage medium of claim 14 , wherein in a case that a size of the current video block is 4*4 or 8*8, the region is corresponding to the first 8 coefficients in a top-left 4*4 coding group.
19. The non-transitory computer-readable storage medium of claim 14 , wherein in a case that a height or width of the current video block are greater than or equal to 4, and a size of the current video block is not 4*4 and 8*8, the region is corresponding to a top-left 4*4 coding group.
20. The apparatus of claim 9 , wherein the side information is included in the bitstream in a case that the last non-zero coefficient is located in a region of the current video block to which the secondary transform tool is applied.