Method and apparatus for generating quantization matrices in video encoding and decoding
When a 2D separable transform is used in video encoding and decoding, two 1D quantization matrices (QMs) are designed based on the horizontal transform and vertical transform of the 2D separable transform. The choice of a 1D QM can be based on whether the horizontal or vertical transform is transform skip, or based on the type of horizontal or vertical transform. Then a 2D QM can be obtained from the product of those two 1D QMs. In a different implementation, an initial 2D QM may be modified based on the 1D QMs. The 2D QMs can be pre-computed and stored in a look-up table for the encoder and decoder. During quantization and de-quantization, the 2D QM to be used for a block can be computed on the fly or retrieved from the look-up table based on the 2D separable transform that is used for the block.
1. A method for video decoding, comprising:
accessing transform coefficients of a block of an image in a video;
de-quantizing said transform coefficients based on a quantization matrix;
inverse transforming said de-quantized transform coefficients based on a separable transform to form decoded prediction residuals, wherein said separable transform corresponds to at least a first transform and a second transform, and wherein said quantization matrix is based on a matrix product of (1) a first vector corresponding to a transform type of said first transform and (2) a second vector corresponding to a transform type of said second transform, said first and second vectors respectively specifying scaling values to be applied to a row and a column of said transform coefficients; and
decoding said block based on said decoded prediction residuals.
2. The method of claim 1 , wherein coefficients of said matrix product are scaled by respective scaling factors to obtain said quantization matrix.
3. The method of claim 2 , wherein said respective scaling factors are based on a power function of a value and corresponding locations of said coefficients.
4. The method of claim 1 , when said first transform is transform skip, all elements in a row of said quantization matrix are same, or when said second transform is transform skip, all elements in a column of said quantization matrix are same.
5. A method for video encoding, comprising:
accessing prediction residuals of a block of an image in a video;
transforming said prediction residuals based on a separable transform to form transformed coefficients, wherein said separable transform corresponds to at least a first transform and a second transform;
quantizing said transformed coefficients based on a quantization matrix, wherein said quantization matrix is based on a matrix product of (1) a first vector corresponding to a transform type of said first transform and (2) a second vector corresponding to a transform type of said second transform, said first and second vectors respectively specifying scaling values to be applied to a row and a column of said transform coefficients; and
encoding said quantized transformed coefficients.
6. The method of claim 5 , wherein coefficients of said matrix product are scaled by respective scaling factors to obtain said quantization matrix.
7. The method of claim 6 , wherein said respective scaling factors are based on a power function of a value and corresponding locations of said coefficients.
8. The method of claim 5 , when said first transform is transform skip, all elements in a row of said quantization matrix are same, or when said second transform is transform skip, all elements in a column of said quantization matrix are same.
9. An apparatus for decoding video data, comprising:
one or more processors, wherein said one or more processors are configured to:
access transform coefficients of a block of an image in a video;
de-quantize said transform coefficients based on a quantization matrix;
inverse transform said de-quantized transform coefficients based on a separable transform to form decoded prediction residuals, wherein said separable transform corresponds to a first transform and a second transform, and wherein said quantization matrix is based on a matrix product of (1) a first vector corresponding to a transform type of said first transform and (2) a second vector corresponding to a transform type of said second transform, said first and second vectors respectively specifying scaling values to be applied to a row and a column of said transform coefficients; and
decode said block based on said decoded prediction residuals.
10. The apparatus of claim 9 , wherein coefficients of said matrix product are scaled by respective scaling factors to obtain said quantization matrix.
11. The apparatus of claim 10 , wherein said respective scaling factors are based on a power function of a value and corresponding locations of said coefficients.
12. The apparatus of claim 9 , when said first transform is transform skip, all elements in a row of said quantization matrix are same, or when said second transform is transform skip, all elements in a column of said quantization matrix are same.
13. An apparatus for encoding video data, comprising:
one or more processors, wherein said one or more processors are configured to:
access prediction residuals of a block of an image in a video;
transform said prediction residuals based on a separable transform to form transform coefficients, wherein said separable transform corresponds to at least a first transform and a second transform;
quantize said transform coefficients based on a quantization matrix, and wherein said quantization matrix is based on a matrix product of (1) a first vector corresponding to a transform type of said first transform and (2) a second vector corresponding to a transform type of said second transform, said first and second vectors respectively specifying scaling values to be applied to a row and a column of said transform coefficients; and
encode said quantized transform coefficients.
14. The apparatus of claim 13 , wherein coefficients of said matrix product are scaled by respective scaling factors to obtain said quantization matrix.
15. The apparatus of claim 14 , wherein said respective scaling factors are based on a power function of a value and corresponding locations of said coefficients.
16. The apparatus of claim 13 , when said first transform is transform skip, all elements in a row of said quantization matrix are same, or when said second transform is transform skip, all elements in a column of said quantization matrix are same.
17. The method of claim 1 , wherein a size of said block is M×N, M being a width and N being a height of said block, wherein said first transform is N-point and said second transform is M-point, and said first vector containing N elements and said second vector containing M elements.
18. The method of claim 5 , wherein a size of said block is M×N, M being a width and N being a height of said block, wherein said first transform is N-point and said second transform is M-point, and said first vector containing N elements and said second vector containing M elements.
19. The apparatus of claim 9 , wherein a size of said block is M×N, M being a width and N being a height of said block, wherein said first transform is N-point and said second transform is M-point, and said first vector containing N elements and said second vector containing M elements.
20. The apparatus of claim 13 , wherein a size of said block is M×N, M being a width and N being a height of said block, wherein said first transform is N-point and said second transform is M-point, and said first vector containing N elements and said second vector containing M elements.