Encoding method and apparatus

1. A video processing method, comprising:

obtaining prediction information of a coding unit (CU);

obtaining, when the CU comprises only one residual transform unit (TU) and when a first size of the residual TU is less than a second size of the CU, a TU partitioning mode of the CU and a residual position of the residual TU;

obtaining transform coefficients of the residual TU based on the horizontal transform type and the vertical transform type, wherein the horizontal transform type and the vertical transform type are based on a TU partitioning mode and the residual position; and

generating a bitstream that comprises the prediction information, a first flag that indicates the TU partitioning mode, a second flag that indicates the residual position, and the transform coefficients.

2. The video processing method of claim 1 , wherein the horizontal transform type is based on discrete sine transform (DST) type VII (DST-7) and the vertical transform type is based on the DST-7, the horizontal transform type is based on the DST-7 and the vertical transform type is based on discrete cosine transform (DCT) type VIII (DCT-8), or the horizontal transform type is based on the DCT-8 and the vertical transform type is based on the DST-7.

3. The video processing method of claim 2 , wherein:

the TU partitioning mode specifies that the CU is partitioned into two first TUs that are arranged horizontally and that comprise a left TU and a right TU, wherein a third size of the left TU is smaller than, greater than, or equal to a fourth size of the right TU; or

the TU partitioning mode specifies that the CU is partitioned into two second TUs that are arranged vertically and that comprise a top TU and a bottom TU, wherein a fifth size of the top TU is smaller than, greater than, or equal to a sixth size of the bottom TU.

4. The video processing method of claim 3 , further comprising:

partitioning the CU into the two first TUs, wherein the residual position is left or right; or

partitioning the CU into the two second TUs, wherein the residual position is a top or a bottom.

5. The video processing method of claim 4 , wherein the horizontal transform type is the DST-7 and the vertical transform type is the DCT-8 when the CU is partitioned into the two second TUs and the residual position is the top, the horizontal transform type is the DST-7 and the vertical transform type is the DST-7 when the CU is partitioned into the two second TUs and the residual position is the bottom, the horizontal transform type is the DCT-8 and the vertical transform type is the DST-7 when the CU is partitioned into the two first TUs and the residual position is the left, and the horizontal transform type is the DST-7 and the vertical transform type is the DST-7 when the CU is partitioned into the two first TUs and the residual position is the right.

6. The video processing method of claim 1 , wherein the residual TU comprises one or more residuals for one or more of a first chrominance component, a second chrominance component, or a luminance component.

7. The video processing method of claim 6 , wherein the one or more residuals comprise a first residual for the first chrominance component, a second residual for the second chrominance component, and a third residual for the luminance component.

8. A video processing apparatus, comprising:

one or more memories configured to store programming instructions; and

at least one processor coupled to the one or more memories and configured to execute the programming instructions to cause the video processing apparatus to:

obtain prediction information of a coding unit (CU);

obtain, when the CU comprises only one residual transform unit (TU) and when a first size of the residual TU is less than a second size of the CU, a TU partitioning mode of the CU and a residual position of the residual TU;

obtain transform coefficients of the residual TU based on the horizontal transform type and the vertical transform type, wherein the horizontal transform type and the vertical transform type are based on a TU partitioning mode and the residual position; and

generate a bitstream that comprises the prediction information, a first flag that indicates the TU partitioning mode, a second flag that indicates the residual position, and the transform coefficients.

9. The video processing apparatus of claim 8 , wherein the horizontal transform type is based on discrete sine transform (DST) type VII (DST-7) and the vertical transform type is based on the DST-7, the horizontal transform type is based on the DST-7 and the vertical transform type is based on discrete cosine transform (DCT) type VIII (DCT-8), or the horizontal transform type is based on the DCT-8 and the vertical transform type is based on the DST-7.

10. The video processing apparatus of claim 9 , wherein:

the TU partitioning mode specifies that the CU is partitioned into two first TUs that are arranged horizontally and that comprise a left TU and a right TU, wherein a third size of the left TU is smaller than, greater than, or equal to a fourth size of the right TU; or

the TU partitioning mode specifies that the CU is partitioned into two second TUs that are arranged vertically and that comprise a top TU and a bottom TU, wherein a fifth size of the top TU is smaller than, greater than, or equal to a sixth size of the bottom TU.

11. The video processing apparatus of claim 10 , wherein when executed by the at least one processor, the programming instructions are further configured to cause the video processing apparatus to:

partition the CU into the two first TUs, wherein the residual position is left or right; or

partition the CU into the two second TUs, wherein the residual position is a top or a bottom.

12. The video processing apparatus of claim 11 , wherein the horizontal transform type is the DST-7 and the vertical transform type is the DCT-8 when the CU is partitioned into the two second TUs and the residual position is the top, the horizontal transform type is the DST-7 and the vertical transform type is the DST-7 when the CU is partitioned into the two second TUs and the residual position is the bottom, the horizontal transform type is the DCT-8 and the vertical transform type is the DST-7 when the CU is partitioned into the two first TUs and the residual position is the left, and the horizontal transform type is the DST-7 and the vertical transform type is the DST-7 when the CU is partitioned into the two first TUs and the residual position is the right.

13. The video processing apparatus of claim 8 , wherein the residual TU comprises one or more residuals for one or more of a first chrominance component, a second chrominance component, or a luminance component.

14. The video processing apparatus of claim 13 , wherein the one or more residuals comprise a first residual for the first chrominance component, a second residual for the second chrominance component, and a third residual for the luminance component.

15. A non-transitory storage medium storing a bitstream that is generated by:

obtaining prediction information of a coding unit (CU);

obtaining, when the CU comprises only one residual transform unit (TU) and when a first size of the residual TU is less than a second size of the CU, a TU partitioning mode of the CU and a residual position of the residual TU;

obtaining transform coefficients of the residual TU based on the horizontal transform type and the vertical transform type, wherein the horizontal transform type and the vertical transform type are based on a TU partitioning mode and the residual position; and

generating the bitstream, wherein the bitstream comprises the prediction information, a first flag that indicates the TU partitioning mode, a second flag that indicates the residual position, and the transform coefficients.

16. The non-transitory storage medium of claim 15 , wherein the horizontal transform type is based on discrete sine transform (DST) type VII (DST-7) and the vertical transform type is based on the DST-7, the horizontal transform type is based on the DST-7 and the vertical transform type is based on discrete cosine transform (DCT) type VIII (DCT-8), or the horizontal transform type is based on the DCT-8 and the vertical transform type is based on the DST-7.

17. The non-transitory storage medium of claim 16 , wherein:

the TU partitioning mode specifies that the CU is partitioned into two first TUs that are arranged horizontally and that comprise a left TU and a right TU, wherein a third size of the left TU is smaller than, greater than, or equal to a fourth size of the right TU; or

the TU partitioning mode specifies that the CU is partitioned into two second TUs that are arranged vertically and that comprise a top TU and a bottom TU, wherein a fifth size of the top TU is smaller than, greater than, or equal to a sixth size of the bottom TU.

18. The non-transitory storage medium of claim 17 , wherein the bitstream is further generated by:

partitioning the CU into the two second TUs, wherein the residual position is left or right; or

partitioning the CU into the two first TUs, wherein the residual position is a top or a bottom.

19. The non-transitory storage medium of claim 18 , wherein the horizontal transform type is the DST-7 and the vertical transform type is the DCT-8 when the CU is partitioned into the two second TUs and the residual position is the top, the horizontal transform type is the DST-7 and the vertical transform type is the DST-7 when the CU is partitioned into the two second TUs and the residual position is the bottom, the horizontal transform type is the DCT-8 and the vertical transform type is the DST-7 when the CU is partitioned into the two first TUs and the residual position is the left, and the horizontal transform type is the DST-7 and the vertical transform type is the DST-7 when the CU is partitioned into the two first TUs and the residual position is the right.

20. The non-transitory storage medium of claim 15 , wherein the residual TU comprises one or more residuals for one or more of a first chrominance component, a second chrominance component, or a luminance component.