Image coding method based on secondary transform and device therefor

1. An image decoding method performed by a decoding apparatus, the method comprising:

receiving, from a bitstream, quantized transform coefficients and a transform index related to a transform;

deriving transform coefficients through dequantization based on the quantized transform coefficients for a target block;

deriving modified transform coefficients based on an inverse reduced secondary transform (RST) using a transform kernel matrix for the transform coefficients;

deriving residual samples for the target block based on an inverse primary transform for the modified transform coefficients; and

generating a reconstructed picture based on the residual samples for the target block,

wherein deriving the modified transform coefficients comprises:

determining a transform set based on an intra prediction mode;

selecting one transform kernel matrix among two transform kernel matrices comprised in the transform set; and

deriving 16 of the modified transform coefficients by applying the selected transform kernel matrix to 8 of the transform coefficients in a 4×4 region of the target block.

2. The image decoding method of claim 1 , wherein when performing a matrix operation between the transform coefficients of the 4×4 region and the transform kernel matrix, 8 of the transform coefficients are one-dimensionally arranged according to a forward diagonal scanning order.

3. The image decoding method of claim 2 , wherein the transform coefficients arranged one-dimensional are two-dimensionally arranged in the 4×4 region according to a row-first direction or a column-first direction corresponding to the intra prediction mode applied to the target block after the matrix operation with the transform kernel matrix.

4. The image decoding method of claim 1 , wherein the transform kernel matrix is a 16×8 matrix, and

wherein a matrix operation between the transform kernel matrix and the 8 of the transform coefficients in the 4×4 region is (16×8 matrix)*(8×1 transform coefficient vector).

5. The image decoding method of claim 4 , wherein when the number of the transform sets is 4 and two transform kernel matrices are included in each transform set, the transform kernel matrix applied to the transform coefficients of the 4×4 region of the target block is as follows:

{transform set=0

{transform index=1

{108,−44,−15,1,−44,19,7,−1,−11,6,2,−1,0,−1,−1,0},

{−40,−97,56,12,−11,29,−12,−3,18,18,−15,−3,−1,−3,2,1},

{25,−31,−1,7,100,−16,−29,1,−54,21,14,−4,−7,2,4,0},

{−32,−39,−92,51,−6,−16,36,−8,3,22,18,−15,4,1,−5,2},

{8,−9,33,−8,−16,−102,36,23,−4,38,−27,−5,5,16,−8,−6},

{−25,5,16,−3,−38,14,11,−3,−97,7,26,1,55,−10,−19,3},

{8,9,16,1,37,36,94,−38,−7,3,−47,11,−6,−13,−17,10},

{2,34,−5,1,−7,24,−25,−3,8,99,−28,−29,6,−43,21,11},

{−16,−27,−39,−109,6,10,16,24,3,19,10,24,−4,−7,−2,−3},

{−9,−10,−34,4,−9,−5,−29,5,−33,−26,−96,33,14,4,39,−14},

{−13,1,4,−9,−30,−17,−3,−64,−35,11,17,19,−86,6,36,14},

{8,−7,−5,−15,7,−30,−28,−87,31,4,4,33,61,−5,−17,22},

{−2,13,−6,−4,−2,28,−13,−14,−3,37,−15,−3,−2,107,−36,−24},

{4,9,11,31,4,9,16,19,12,33,32,94,12,0,34,−45},

{2,−2,8,−16,8,5,28,−17,6,−7,18,−45,40,36,97,−8},

{0,−2,0,−10,−1,−7,−3,−35,−1,−7,−2,−32,−6,−33,−16,−112},

}

{transform index=2

{119,−30,−22,−3,−23,−2,3,2,−16,3,6,0,−3,2,1,0},

{−27,−101,31,17,−47,2,22,3,19,30,−7,−9,5,3,−5,−1},

{0,58,22,−15,−102,2,38,2,10,−13,−5,4,14,−1,−9,0},

{23,4,66,−11,22,89,−2,−26,13,−8,−38,−1,−9,−20,−2,8},

{−19,−5,−89,2,−26,76,−11,−17,20,13,18,−4,1,−15,3,5},

{−10,−1,−1,6,23,25,87,−7,−74,4,39,−5,0,−1,−20,−1},

{−17,−28,12,−8,−32,14,−53,−6,−68,−67,17,29,2,6,25,4},

{1,−24,−23,1,17,−7,52,9,50,−92,−15,27,−15,−10,−6,3},

{−6,−17,−2,−111,7,−17,8,−42,9,18,16,25,−4,2,−1,11},

{9,5,35,0,6,21,−9,34,44,−3,102,11,−7,13,11,−20},

{4,−5,−5,−10,15,19,−2,6,6,−12,−13,6,95,69,−29,−24},

{−6,−4,−9,−39,1,22,0,102,−19,19,−32,30,−16,−14,−8,−23},

{4,−4,7,8,4,−13,−18,5,0,0,21,22,58,−88,−54,28},

{−4,−7,0,−24,−7,0,−25,3,−3,−30,8,−76,−34,4,−80,−26},

{0,6,0,30,−6,1,−13,−23,1,20,−2,80,−44,37,−68,1},

{0,0,−1,5,−1,−7,1,−34,−2,3,−6,19,5,−38,11,−115},

}

},

{transform set=1

{transform index=1

{−111,39,4,3,44,11,−12,−1,7,−16,−5,2,3,−1,4,2},

{−47,−27,15,−1,−92,43,20,−2,20,39,−16,−5,10,−5,−13,2},

{−35,−23,4,4,−17,−72,32,6,−59,18,50,−6,0,40,0,−13},

{13,93,−27,−4,−48,13,−34,4,−52,11,1,10,3,16,−3,1},

{−11,−27,1,2,−47,−4,−36,10,−2,−85,14,29,−20,−2,57,4},

{0,−35,32,−2,26,60,−3,−17,−82,1,−30,0,−37,21,3,12},

{−17,−46,−92,14,7,−10,−39,29,−17,27,−28,17,1,−15,−13,17},

{4,−10,−23,4,16,58,−17,26,30,21,67,2,−13,59,13,−40},

{5,−20,32,−5,8,−3,−46,−7,−4,2,−15,24,100,44,0,5},

{−4,−1,38,−18,−7,−42,−63,−6,33,34,−23,15,−65,33,−20,2},

{−2,−10,35,−19,5,8,−44,14,−25,25,58,17,7,−84,−16,−18},

{5,13,18,34,11,−4,18,18,5,58,−3,42,−2,−10,85,38},

{−5,−7,−34,−83,2,−1,−4,−73,4,20,15,−12,4,−3,44,12},

{0,4,−2,−60,5,9,42,34,5,−14,9,80,−5,13,−38,37},

{−1,2,7,−57,3,−7,9,68,−9,6,−49,−20,6,−4,36,−64},

{−1,0,−12,23,1,−4,17,−53,−3,4,−21,72,−4,−8,−3,−83},

}

{transform index=2

{88,−55,6,−3,−66,27,9,−2,11,11,−13,1,−2,−7,1,2},

{−58,−20,27,−2,−27,75,−29,0,47,−42,−11,11,−9,−3,19,−4},

{−51,23,−22,5,−63,3,37,−5,1,64,−35,−4,29,−31,−11,13},

{−27,−76,49,−2,40,14,9,−17,−56,36,−25,6,14,3,−6,8},

{19,−4,−36,22,52,7,36,−23,28,−17,−64,15,−5,−44,48,9},

{29,50,13,−10,1,34,−59,1,−51,4,−16,30,52,−33,24,−5},

{−12,−21,−74,43,−13,39,18,−5,−58,−35,27,−5,19,26,6,−5},

{19,38,−10,−5,28,66,0,−5,−4,19,−30,−26,−40,28,−60,37},

{−6,27,18,−5,−37,−18,12,−25,−44,−10,−38,37,−66,45,40,−7},

{−13,−28,−45,−39,0,−5,−39,69,−23,16,−12,−18,−50,−31,24,13},

{−1,8,24,−51,−15,−9,44,10,−28,−70,−12,−39,24,−18,−4,51},

{−8,−22,−17,33,−18,−45,−57,−27,0,−31,−30,29,−2,−13,−53,49},

{1,12,32,51,−8,8,−2,−31,−22,4,46,−39,−49,−67,14,17},

{4,5,24,60,−5,−14,−23,38,9,8,−34,−59,24,47,42,28},

{−1,−5,−20,−34,4,4,−15,−46,18,31,42,10,10,27,49,78},

{−3,−7,−22,−34,−5,−11,−36,−69,−1,−3,−25,−73,5,4,4,−49},

}

},

{transform set=2

{transform index=1

{−112,47,−2,2,−34,13,2,0,15,−7,1,0,8,−3,−1,0},

{29,−7,1,−1,−108,40,2,0,−45,13,4,−1,8,−5,1,0},

{−36,−87,69,−10,−17,−33,26,−2,7,14,−11,2,6,8,−7,0},

{28,−5,2,−2,−29,13,−2,0,103,−36,−4,1,48,−16,−4,1},

{−12,−24,15,−3,26,80,−61,9,15,54,−36,2,0,−4,6,−2},

{18,53,69,−74,14,24,28,−30,−6,−7,−11,12,−5,−7,−6,8},

{5,−1,2,0,−26,6,0,1,45,−9,−1,0,−113,28,8,−1},

{−13,−32,18,−2,15,34,−27,7,−25,−80,47,−1,−16,−50,28,2},

{−4,−13,−10,19,18,46,60,−48,16,33,60,−48,1,0,5,−2},

{15,33,63,89,8,15,25,40,−4,−8,−15,−8,−2,−6,−9,−7},

{−8,−24,−27,15,12,41,26,−29,−17,−50,−39,27,0,35,−67,26},

{−2,−6,−24,13,−1,−8,37,−22,3,18,−51,22,−23,−95,17,17},

{−3,−7,−16,−21,10,24,46,75,8,20,38,72,1,2,1,7},

{2,6,10,−3,−5,−16,−31,12,7,24,41,−16,−16,−41,−89,49},

{4,8,21,40,−4,−11,−28,−57,5,14,31,70,7,18,32,52},

{0,1,4,11,−2,−4,−13,−34,3,7,20,47,−6,−19,−42,−101},

}

{transform index=2

{−99,39,−1,2,65,−20,−5,0,−15,−2,5,−1,0,3,−1,0},

{58,42,−33,3,33,−63,23,−1,−55,32,3,−5,21,−2,−8,3},

{−15,71,−44,5,−58,−29,25,3,62,−7,−4,−4,−19,4,0,1},

{46,5,4,−6,71,−12,−15,5,52,−38,13,−2,−63,23,3,−3},

{−14,−54,−29,29,25,−9,61,−29,27,44,−48,5,−27,−21,12,7},

{−3,3,69,−42,−11,−50,−26,26,24,63,−19,−5,−18,−22,12,0},

{17,16,−2,1,38,18,−12,0,62,1,−14,5,89,−42,8,−2},

{15,54,−8,6,6,60,−26,−8,−30,17,−38,22,−43,−45,42,−7},

{−6,−17,−55,−28,9,30,−8,58,4,34,41,−52,−16,−36,−20,16},

{−2,−1,−9,−79,7,11,48,44,−13,−34,−55,6,12,23,20,−11},

{7,29,14,−6,12,53,10,−11,14,59,−15,−3,5,71,−54,13},

{−5,−24,−53,15,−3,−15,−61,26,6,30,−16,23,13,56,44,−35},

{4,8,21,52,−1,−1,−5,29,−7,−17,−44,−84,8,20,31,39},

{−2,−11,−25,−4,−4,−21,−53,2,−5,−26,−64,19,−8,−19,−73,39},

{−3,−5,−23,−57,−2,−4,−24,−75,1,3,9,−25,6,15,41,61},

{1,1,7,18,1,2,16,47,2,5,24,67,3,9,25,88},

}

},

{transform set=3

{transform index=1

{−114,37,3,2,−22,−23,14,0,21,−17,−5,2,5,2,−4,−1},

{−19,−41,19,−2,85,−60,−11,7,17,31,−34,2,−11,19,2,−8},

{36,−25,18,−2,−42,−53,35,5,46,−60,−25,19,8,21,−33,−1},

{−27,−80,44,−3,−58,1,−29,19,−41,18,−12,−7,12,−17,7,−6},

{−11,−21,37,−10,44,−4,47,−12,−37,−41,58,18,10,−46,−16,31},

{15,47,10,−6,−16,−44,42,10,−80,25,−40,21,−23,−2,3,−14},

{13,25,79,−39,−13,10,31,−4,49,45,12,−8,3,−1,43,7},

{16,11,−26,13,−13,−74,−20,−1,5,−6,29,−47,26,−49,54,2},

{−8,−34,−26,7,−26,−19,29,−37,1,22,46,−9,−81,37,14,20},

{−6,−30,−42,−12,−3,5,57,−52,−2,37,−12,6,74,10,6,−15},

{5,9,−6,42,−15,−18,−9,26,15,58,14,43,23,−10,−37,75},

{−5,−23,−23,36,3,22,36,40,27,−4,−16,56,−25,−46,56,−24},

{1,3,23,73,8,5,34,46,−12,2,35,−38,26,52,2,−31},

{−3,−2,−21,−52,1,−10,−17,44,−19,−20,30,45,27,61,49,21},

{−2,−7,−33,−56,−4,−6,21,63,15,31,32,−22,−10,−26,−52,−38},

{−5,−12,−18,−12,8,22,38,36,−5,−15,−51,−63,−5,0,15,73},

},

{transform index=2

{−102,22,7,2,66,−25,−6,−1,−15,14,1,−1,2,−2,1,0},

{12,93,−27,−6,−27,−64,36,6,13,5,−23,0,−2,6,5,−3},

{−59,−24,17,1,−62,−2,−3,2,83,−12,−17,−2,−24,14,7,−2},

{−33,23,−36,11,−21,50,35,−16,−23,−78,16,19,22,15,−30,−5},

{0,−38,−81,30,27,5,51,−32,24,36,−16,12,−24,−8,9,1},

{28,38,8,−9,62,32,−13,2,51,−32,15,5,−66,28,0,−1},

{11,−35,21,−17,30,−18,31,18,−11,−36,−80,12,16,49,13,−32

{−13,23,22,−36,−12,64,39,25,−19,23,−36,9,−30,−58,33,−7},

{−9,−20,−55,−83,3,−2,1,62,8,2,27,−28,7,15,−11,5},

{−6,24,−38,23,−8,40,−49,0,−7,9,−25,−44,23,39,70,−3},

{12,17,17,0,32,27,21,2,67,11,−6,−10,89,−22,−12,16},

{2,−9,8,45,7,−8,27,35,−9,−31,−17,−87,−23,−22,−19,44},

{−1,−9,28,−24,−1,−10,49,−30,−8,−7,40,1,4,33,65,67},

{5,−12,−24,−17,13,−34,−32,−16,14,−67,−7,9,7,−74,49,1},

{2,−6,11,45,3,−10,33,55,8,−5,59,4,7,−4,44,−66},

{−1,1,−14,36,−1,2,−20,69,0,0,−15,72,3,4,5,65},

}

}.

6. An image encoding method performed by an image encoding apparatus, the method comprising:

deriving prediction samples based on an intra prediction mode applied to a target block;

deriving residual samples for the target block based on the prediction samples;

deriving transform coefficients for the target block based on a primary transform for the residual samples;

deriving modified transform coefficients based on a reduced secondary transform (RST) using a predetermined matrix kernel matrix for the transform coefficients; and

deriving quantized transform coefficients by performing quantization based on the modified transform coefficients,

wherein deriving the modified transform coefficients comprises:

determining a transform set based on an intra prediction mode;

selecting one transform kernel matrix among two transform kernel matrices comprised in the transform set; and

deriving 8 of the modified transform coefficients by applying the selected transform kernel matrix to 16 of the transform coefficients in a 4×4 region of the target block.

7. The image encoding method of claim 6 , wherein the transform coefficients of the 4×4 region are one-dimensionally arranged according to a row-first direction or a column-first direction corresponding to the intra prediction mode applied to the target block.

8. The image encoding method of claim 7 , wherein the derived modified transform coefficients are arranged according to a diagonal scanning direction in a top-left region of the 4×4 region.

9. The image encoding method of claim 7 , wherein the transform kernel matrix is a 8×16 matrix, and

wherein a matrix operation between the transform kernel matrix and the 16 of the transform coefficients in the 4×4 region is (8×16 matrix)*(16×1 transform coefficient vector).

10. A non-transitory computer readable storage medium storing a bitstream generated by a method, the method comprising:

deriving prediction samples based on an intra prediction mode applied to a target block;

deriving residual samples for the target block based on the prediction samples;

deriving transform coefficients for the target block based on a primary transform for the residual samples;

deriving modified transform coefficients based on a reduced secondary transform (RST) using a predetermined matrix kernel matrix for the transform coefficients;

deriving quantized transform coefficients by performing quantization based on the modified transform coefficients, and

encoding image information including information related to the quantized transform coefficients to generate the bitstream,

wherein deriving the modified transform coefficients comprises:

determining a transform set based on an intra prediction mode;

selecting one transform kernel matrix among two transform kernel matrices comprised in the transform set and

deriving 8 of the modified transform coefficients by applying the selected transform kernel matrix to 16 of the transform coefficients in a 4×4 region of the target block.