Method and apparatus of matrix based intra prediction in image and video processing
An image or video processing method comprises receiving input data of a current block, mapping an MIP mode index to a default mode, and encoding or decoding the current block by referencing the default mode. Another image or video processing method comprises receiving input data of a current block coded or to be coded by an MIP mode, and parsing or signaling a transpose flag for the current block indicating whether transposing is applied in coding the current block. Reference samples of the current block are prepared from neighboring boundary samples according to the transpose flag. A matrix is selected to be multiplied to the reference samples to generate predicted samples. A transposing process is adaptively applied to the current block according to the transpose flag, and the predicted samples are used to encode or decode the current block.
1. A method of processing video data in an image or video encoding system, comprising:
receiving input data associated with a current block to be coded by a Matrix-based Intra Prediction (MIP) mode in a current picture;
determining whether transposing is applied in coding the current block by the MIP mode;
signaling a transpose flag for the current block indicating whether transposing is applied based on the determining;
deriving reference samples of the current block from neighboring boundary samples of the current block in the current picture according to whether transposing is applied;
selecting a matrix for the current block from a matrix set of the current block based on information about the MIP mode, wherein the transpose flag is signaled before the information about the MIP mode is signaled;
generating predicted samples of the current block by multiplying the reference samples and the matrix;
adaptively performing a transposing process by replacing the predicted samples of the current block with a transpose of the predicted samples according to whether transposing is applied; and
encoding the current block in the current picture according to the predicted samples of the current block.
2. The method of claim 1 , further comprising clipping the predicted samples to a rational range and up-sampling the clipped predicted samples, wherein the transposing process is adaptively performed on the clipped predicted samples of the current block before up-sampling.
3. The method of claim 1 , further comprising signaling a matrix index for the current block indicating which matrix is selected from the matrix set of the current block, wherein the matrix set is determined based on a size of the current block.
4. The method of claim 3 , wherein MIP mode information of the current block is signaled by first signaling the transpose flag and then signaling the matrix index.
5. The method of claim 1 , wherein the step of deriving reference samples of the current block from neighboring boundary samples of the current block further comprises obtaining top boundary reference samples and left boundary reference samples, reducing the top boundary reference samples and left boundary reference samples into reduced top boundary samples and reduced left boundary samples, and concatenating the reduced top boundary samples and the reduced left boundary samples according to whether transposing is applied.
6. An apparatus for performing image or video processing, comprising: a computer processor configured for receiving input data and programming executable on the computer processor for image or video encoding by performing steps comprising:
receiving input data associated with a current block to be coded by a Matrix-based Intra Prediction (MIP) mode in a current picture;
determining whether transposing is applied in coding the current block by the MIP mode;
signaling a transpose flag for the current block indicating whether transposing is applied based on the determining;
deriving reference samples of the current block from neighboring boundary samples of the current block in the current picture according to whether transposing is applied;
selecting a matrix for the current block from a matrix set of the current block based on information about the MIP mode, wherein the transpose flag is signaled before the information about the MIP mode is signaled;
generating predicted samples of the current block by multiplying the reference samples and the matrix;
adaptively performing a transposing process by replacing the predicted samples of the current block with a transpose of the predicted samples according to whether transposing is applied; and
encoding the current block in the current picture according to the predicted samples of the current block.
7. A method of processing video data in an image or video decoding system, comprising:
receiving input data associated with a current block coded by a Matrix-based Intra Prediction (MIP) mode in a current picture;
parsing a transpose flag for the current block indicating whether transposing is applied;
determining whether transposing is applied in coding the current block by the MIP mode based on the transpose flag;
deriving reference samples of the current block from neighboring boundary samples of the current block in the current picture according to whether transposing is applied;
selecting a matrix for the current block from a matrix set of the current block based on information about the MIP mode, wherein the transpose flag is parsed before the information about the MIP mode is parsed;
generating predicted samples of the current block by multiplying the reference samples and the matrix;
adaptively performing a transposing process by replacing the predicted samples of the current block with a transpose of the predicted samples according to whether transposing is applied; and
decoding the current block in the current picture according to the predicted samples of the current block.
8. The method of claim 7 , further comprising clipping the predicted samples to a rational range and up-sampling the clipped predicted samples, wherein the transposing process is adaptively performed on the clipped predicted samples of the current block before up-sampling.
9. The method of claim 7 , further comprising parsing a matrix index for the current block indicating which matrix is selected from the matrix set of the current block, wherein the matrix set is determined based on a size of the current block.
10. The method of claim 9 , wherein MIP mode information of the current block is parsed by first parsing the transpose flag and then parsing the matrix index.
11. The method of claim 7 , wherein the step of deriving reference samples of the current block from neighboring boundary samples of the current block further comprises obtaining top boundary reference samples and left boundary reference samples, reducing the top boundary reference samples and left boundary reference samples into reduced top boundary samples and reduced left boundary samples, and concatenating the reduced top boundary samples and the reduced left boundary samples according to whether transposing is applied.
12. An apparatus for performing image or video processing, comprising: a computer processor configured for receiving input data and programming executable on the computer processor for image or video decoding by performing steps comprising:
receiving input data associated with a current block coded by a Matrix-based Intra Prediction (MIP) mode in a current picture;
parsing a transpose flag for the current block indicating whether transposing is applied;
determining whether transposing is applied in coding the current block by the MIP mode based on the transpose flag;
deriving reference samples of the current block from neighboring boundary samples of the current block in the current picture according to whether transposing is applied;
selecting a matrix for the current block from a matrix set of the current block based on information about the MIP mode, wherein the transpose flag is parsed before the information about the MIP mode is parsed;
generating predicted samples of the current block by multiplying the reference samples and the matrix;
adaptively performing a transposing process by replacing the predicted samples of the current block with a transpose of the predicted samples according to whether transposing is applied; and
decoding the current block in the current picture according to the predicted samples of the current block.