Determination of secondary transform for intra prediction
A method for video decoding in a decoder is provided. Coding information of a block to be reconstructed is decoded from a coded video bitstream. The coding information indicates intra prediction information for the block. Responsive to the block being coded with a directional mode, the directional mode is determined based on a nominal mode and an angular offset, the coding information indicating the nominal mode and the angular offset, a first non-separable transform set of one or more non-separable transforms for the block is determined based on the nominal mode, a non-separable transform in the first non-separable transform set is determined based on a non-separable transform index indicated by the coding information, and the block is reconstructed based on the directional mode and the non-separable transform.
1. A method for video decoding in a decoder, comprising:
decoding coding information of a block to be reconstructed from a coded video bitstream, the coding information indicating intra prediction information for the block; and
when the block is coded with a directional mode,
determining the directional mode based on a nominal mode and an angular offset, the coding information indicating the nominal mode and the angular offset,
determining a first non-separable transform set of one or more non-separable transforms for the block from a plurality of non-separable transform sets based on a nominal direction indicated by the nominal mode, each of the non-separable transform sets corresponding to a respective one of 8 nominal directions, the 8 nominal directions are associated with V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED,
determining a non-separable transform in the first non-separable transform set based on a non-separable transform index indicated by the coding information, and
reconstructing the block based on the directional mode and the non-separable transform.
2. The method of claim 1 , wherein the determining the first non-separable transform set further comprises:
determining a transform set mode that is associated with the nominal mode,
the transform set mode indicating the first non-separable transform set that includes the non-separable transform.
3. The method of claim 1 , wherein the non-separable transform is a non-separable secondary transform.
4. The method of claim 3 , wherein the non-separable secondary transform does not apply to a PAETH mode or a recursive filtering mode.
5. The method of claim 3 , wherein the reconstructing the block based on the directional mode and the non-separable secondary transform further comprises:
applying the non-separable secondary transform only to N first transform coefficients along a scanning order used for entropy coding the first transform coefficients of the block.
6. The method of claim 3 , wherein the reconstructing the block based on the directional mode and the non-separable secondary transform further comprises:
applying the non-separable secondary transform only to first transform coefficients in the block, each of the first transform coefficients having a coordinate (x, y) and a sum of the respective x and y coordinates being less than a threshold value.
7. The method of claim 3 , wherein a horizontal transform and a vertical transform in a primary transform for the block are included in a subset of a set of line graph transforms.
8. The method of claim 3 , wherein
the block includes first transform coefficients obtained with the non-separable secondary transform and second transform coefficients obtained without the non-separable secondary transform; and
the reconstructing the block further includes entropy decoding the first transform coefficients and the second transform coefficients separately.
9. The method of claim 1 , wherein
non-directional modes include a DC mode, a PAETH mode, a SMOOTH mode, a SMOOTH_V mode, a SMOOTH_H mode, recursive filtering modes, and a chroma from luma (CfL) mode, the DC mode, the PAETH mode, the SMOOTH mode, the SMOOTH_V mode, and the SMOOTH_H mode being based on averaging of neighboring samples of the block, and
when the block is coded with one of the non-directional modes,
determining a second non-separable transform set of one or more non-separable transforms associated with the one of the non-directional modes, one of (a) at least another one of the non-directional modes and (b) a nominal mode being associated with the second non-separable transform set,
determining a non-separable transform in the second non-separable transform set based on the non-separable transform index indicated by the coding information, and
reconstructing the block based on the non-directional mode and the non-separable transform.
10. The method of claim 9 , wherein the one of the non-directional modes and the one of the at least another one of the non-directional modes and the nominal mode include one of (a) the recursive filtering modes and one of the DC mode and the SMOOTH mode, (b) the SMOOTH mode, the SMOOTH_H mode, and the SMOOTH_V mode, (c) the SMOOTH mode, the SMOOTH_H mode, the SMOOTH_V mode, and the PAETH mode, (d) the recursive filtering modes, the SMOOTH mode, and the PAETH mode, (e) a vertical mode for the nominal mode and the SMOOTH_V mode, (f) a horizontal mode for the nominal mode and the SMOOTH_H mode, and (v) the CfL mode and one of the DC mode, the SMOOTH mode, and the PAETH mode.
11. A method for video encoding in an encoder, comprising:
when a block is coded with a directional mode,
determining the directional mode based on a nominal mode and an angular offset,
determining a first non-separable transform set of one or more non-separable transforms for the block from a plurality of non-separable transform sets based on a nominal direction indicated by the nominal mode, each of the non-separable transform sets corresponding to a respective one of 8 nominal directions, the 8 nominal directions are associated with V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67 PRED,
determining a non-separable transform in the first non-separable transform set associated with a non-separable transform index,
reconstructing the block based on the directional mode and the non-separable transform, and
encoding coding information indicating the nominal mode, the angular offset, and the non-separable transform index in a bitstream.
12. The method of claim 11 , wherein the determining the first non-separable transform set further comprises:
determining a transform set mode that is associated with the nominal mode,
the transform set mode indicating the first non-separable transform set that includes the non-separable transform.
13. The method of claim 11 , wherein the non-separable transform is a non-separable secondary transform.
14. The method of claim 3 , wherein the non-separable secondary transform does not apply to a PAETH mode or a recursive filtering mode.
15. The method of claim 13 , wherein the reconstructing the block based on the directional mode and the non-separable secondary transform further comprises:
applying the non-separable secondary transform only to N first transform coefficients along a scanning order used for entropy coding the first transform coefficients of the block.
16. The method of claim 13 , wherein the reconstructing the block based on the directional mode and the non-separable secondary transform further comprises:
applying the non-separable secondary transform only to first transform coefficients in the block, each of the first transform coefficients having a coordinate (x, y) and a sum of the respective x and y coordinates being less than a threshold value.
17. The method of claim 13 , wherein a horizontal transform and a vertical transform in a primary transform for the block are included in a subset of a set of line graph transforms.
18. The method of claim 13 , wherein
the block includes first transform coefficients obtained with the non-separable secondary transform and second transform coefficients obtained without the non-separable secondary transform; and
the reconstructing the block further includes entropy decoding the first transform coefficients and the second transform coefficients separately.
19. The method of claim 11 , wherein
non-directional modes include a DC mode, a PAETH mode, a SMOOTH mode, a SMOOTH_V mode, a SMOOTH_H mode, recursive filtering modes, and a chroma from luma (CfL) mode, the DC mode, the PAETH mode, the SMOOTH mode, the SMOOTH_V mode, and the SMOOTH_H mode being based on averaging of neighboring samples of the block, and
when the block is coded with one of the non-directional modes,
determining a second non-separable transform set of one or more non-separable transforms associated with the one of the non-directional modes, one of (a) at least another one of the non-directional modes and (b) a nominal mode being associated with the second non-separable transform set,
determining a non-separable transform in the second non-separable transform set associated with the non-separable transform index,
reconstructing the block based on the non-directional mode and the non-separable transform, and
encoding coding information indicating the non-separable transform index in the bitstream.
20. A method of processing visual media data, the method comprising:
processing a bitstream that includes the visual media data according to a format rule, wherein
the bitstream includes coding information of a block to be reconstructed, the coding information indicating intra prediction information for a block; and
the format rule specifies that:
when the block is coded with a directional mode,
the directional mode is determined based on a nominal mode and an angular offset, the coding information indicating the nominal mode and the angular offset,
a first non-separable transform set of one or more non-separable transforms for the block is determined from a plurality of non-separable transform sets based on a nominal direction indicated by the nominal mode, each of the non-separable transform sets corresponding to a respective one of 8 nominal directions, the 8 nominal directions are associated with V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED,
a non-separable transform in the first non-separable transform set is determined based on a non-separable transform index indicated by the coding information, and
the block is processed based on the directional mode and the non-separable transform.