Intra prediction and residual coding

1. A method of processing video data, comprising:

performing, for a conversion between visual media data of a video and a bitstream of the video, a prediction for a current video block of the visual media data;

generating residual samples corresponding to the current video block or a subblock split from the current video block;

quantizing the residual samples to generate quantized residual samples; and

coding the quantized residual samples to perform the conversion,

wherein a first one-dimensional (1-d) transform is applied to the residual samples before the quantizing and/or a second 1-d transform is applied to the quantized residual samples, and

wherein the current video block or the subblock is coded with a block differential pulse-code modulation (BDPCM) scheme and/or a quantized residual domain BDPCM (RBDPCM) scheme.

2. The method of claim 1 , wherein the current video block or the subblock is coded with the RBDPCM scheme.

3. The method of claim 1 , wherein the prediction is performed at least based on one or more intra prediction modes.

4. The method of claim 1 , wherein the first 1-d transform is applied to the residual samples in a first direction to generate transform coefficients, and the first direction depends on a direction in which the residual samples are coded.

5. The method of claim 4 , wherein the first 1-d transform is applied to each row of the residual samples in response to the residual samples being coded on a row-by-row basis, and the first 1-d transform is applied to each column of the residual samples in response to the residual samples being coded on a column-by-column basis.

6. The method of claim 5 , wherein the first direction depends on a direction in which the residual samples are predicted, the first 1-d transform is applied to each row of the residual samples in response to the residual samples being predicted in a vertical direction, and the first 1-d transform is applied to each column of the residual samples in response to the residual samples being predicted in a horizontal direction.

7. The method of claim 2 , wherein the second 1-d transform is applied to the quantized residual samples in a second direction after the residual samples are predicted in the RBDPCM scheme, and the second direction depends on a direction in which the residual samples are predicted in the RBDPCM scheme,

wherein the second 1-d transform is applied to each column of the residual samples in response to the residual samples being predicted in a vertical direction, and the second 1-d transform is applied to each row of the residual samples in response to the residual samples being predicted in a horizontal direction.

8. The method of claim 1 , wherein the second 1-d transform is applied to at least one subset of residual samples, wherein the at least one subset of residual samples comprises more than one line of residual samples in a specific direction.

9. The method of claim 8 , wherein the more than one line of residual samples in the specific direction comprises more than one row of residual samples or more than one column of residual samples, and

wherein the at least one subset of residual samples comprises a first subset of residual samples and a second subset of residual samples, and transform coefficients of the second subset of residual samples are predicted from transform coefficients or quantized transform coefficients of the first subset of residual samples.

10. The method of claim 1 , further comprising:

applying a first quantization to transform coefficients generated by the second 1-d transform and applying a second quantization to the residual samples before the second 1-d transform is performed.

11. The method of claim 1 , wherein differences between transform coefficients to be coded and previous transform coefficients are calculated and quantized in a first quantization when the transform coefficients to be coded are predicted from the previous transform coefficients.

12. The method of claim 3 , wherein a residual prediction direction of the residual samples is different from an intra prediction direction of the current video block or a residual prediction direction of the residual samples is same as an intra prediction direction of the current video block.

13. The method of claim 12 , wherein the residual prediction direction comprises at least one of a horizontal direction, a vertical direction, a 45-degree diagonal direction and a 135-degree diagonal direction.

14. The method of claim 1 , wherein the residual samples are quantized with a dependent quantization (DQ) or trellis coded quantization (TCQ).

15. The method of claim 1 , wherein the residual samples are quantized with different quantization parameters (QPs) and/or different quantization steps, and a QP applied to a specific region in the current video block is different from a QP associated with the current video block.

16. The method of claim 1 , wherein the method further comprising:

modifying the residual samples at a specific location before performing at least one of a quantization and a transform on the residual samples.

17. The method of claim 1 , wherein the residual samples are coded in a transform mode, wherein transform cores used in the transform mode are flipped in response to an indication so that transform coefficients can be scanned in a same order as in a transform skip mode.

18. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

perform, for a conversion between visual media data of a video and a bitstream of the video, a prediction for a current video block of the visual media data;

generate residual samples corresponding to the current video block or a subblock split from the current video block;

quantize the residual samples to generate quantized residual samples; and

code the quantized residual samples to perform the conversion,

wherein a first one-dimensional (1-d) transform is applied to the residual samples before the quantizing and/or a second 1-d transform is applied to the quantized residual samples, and

wherein the current video block or the subblock is coded with a block differential pulse-code modulation (BDPCM) scheme and/or a quantized residual domain BDPCM (RBDPCM) scheme.

19. The apparatus of claim 18 , wherein the current video block or the subblock is coded with the RBDPCM scheme.

20. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

performing a prediction for a current video block of visual media data of the video;

generating residual samples corresponding to the current video block or a subblock split from the current video block;

quantizing the residual samples to generate quantized residual samples; and

generating the bitstream based on the quantized residual samples,

wherein a first one-dimensional (1-d) transform is applied to the residual samples before the quantizing and/or a second 1-d transform is applied to the quantized residual samples, and

wherein the current video block or the subblock is coded with a block differential pulse-code modulation (BDPCM) scheme and/or a quantized residual domain BDPCM (RBDPCM) scheme.