Method and apparatus for performing transform using layered givens transform

1. A method for performing a transform by using Layered Givens Transform (LGT), the method comprising:

deriving at least one rotation layer, a first permutation layer and a second permutation layer based on a given transform matrix H and an error parameter, wherein:

the first permutation layer or the second permutation layer is derived through a first optimization procedure based on Hungarian algorithm,

the first optimization procedure is performed based on matching a LGT matrix with the given transform matrix H,

the LGT matrix is derived by using the at least one rotation layer, the first permutation layer, and the second permutation layer, and

the Hungarian algorithm is performed by using an input to which an absolute operator is applied;

obtaining a LGT coefficient based on the at least one rotation layer, the first permutation layer, and the second permutation layer; and

performing quantization and entropy encoding with respect to the LGT coefficient,

wherein the first permutation layer and the second permutation layer include a permutation matrix obtained by permuting a row of an identity matrix, and

wherein the LGT coefficient is obtained by applying the second permutation layer after applying the first permutation layer and the at least one rotation layer.

2. The method of claim 1 , wherein the at least one rotation layer includes a permutation matrix and

at least one pairwise rotation matrix.

3. The method of claim 1 ,

wherein the at least one rotation layer is derived through a second optimization procedure, and

wherein the second optimization procedure includes:

determining an optimal rotation angle of a rotation matrix in the at least one rotation layer based on matching the LGT matrix with the given transform matrix H, and

determining non-overlapped pairs that maximize total summation of gain based on the optimal rotation angle.

4. The method of claim 3 , wherein the second optimization procedure further includes setting at least one other rotation layer in the LGT to an identity matrix.

5. A method for performing an inverse-transform by using Layered Givens Transform (LGT), the method comprising:

obtaining a LGT coefficient by performing an entropy encoding and a dequantization with respect to a residual signal included in a video signal;

deriving at least one rotation layer, a first permutation layer, and a second permutation layer based on a given transform matrix H and an error parameter, wherein:

the first permutation layer or the second permutation layer is derived through a first optimization procedure based on Hungarian algorithm,

the first optimization procedure is performed based on matching a LGT matrix with the given transform matrix H,

the LGT matrix is derived by using the at least one rotation layer, the first permutation layer, and the second permutation layer, and

the Hungarian algorithm is performed by using an input to which an absolute operator is applied;

performing an inverse-transform with respect to the LGT coefficient based on the at least one rotation layer, the first permutation layer and the second permutation layer; and

reconstructing the video signal by using the inverse-transformed coefficient,

wherein the first permutation layer and the second permutation layer include a permutation matrix obtained by permuting a row of an identity matrix, and

wherein the inverse-transform with respect to the LGT coefficient is performed by applying the first second permutation layer applying the first permutation layer and the at least one rotation layer.

6. The method of claim 5 , wherein the at least one rotation layer includes a permutation matrix and

at least one pairwise rotation matrix.

7. The method of claim 5 ,

wherein the at least one rotation layer is derived through a second optimization procedure, and

wherein the second optimization procedure includes:

determining an optimal rotation angle of a rotation matrix in the at least one rotation layer based on matching the LGT matrix with the given transform matrix H, and

determining non-overlapped pairs that maximize total summation of gain based on the optimal rotation angle.

8. The method of claim 7 , wherein the second optimization procedure further includes setting at least one other rotation layer in the LGT to an identity matrix.

9. An apparatus for performing an inverse-transform by using Layered Givens Transform (LGT), the apparatus comprising:

a processor configured to:

entropy-decode a residual signal included in a video signal,

derive at least one rotation layer, a first permutation layer, and a second permutation layer based on a given transform matrix H and an error parameter, wherein:

the first permutation layer or the second permutation layer is derived through a first optimization procedure based on Hungarian algorithm,

the first optimization procedure is performed based on matching a LGT matrix with the given transform matrix H,

the LGT matrix is derived by using the at least one rotation layer, the first permutation layer, and the second permutation layer, and

the Hungarian algorithm is performed by using an input to which an absolute operator is applied,

obtain a LGT coefficient based on the at least one rotation layer, the first permutation layer, and the second permutation layer, and

perform quantization and entropy encoding with respect to the LGT coefficient,

wherein the first permutation layer and the second permutation layer include a permutation matrix obtained by permuting a row of an identity matrix, and

wherein the inverse-transform with respect to the LGT coefficient is performed by applying the second permutation layer after applying the first permutation layer and the at least one rotation layer.