Apparatus and method for applying artificial neural network to image encoding or decoding
The present disclosure relates to video encoding or decoding and, more specifically, to an apparatus and a method for applying an artificial neural network (ANN) to video encoding or decoding. The apparatus and the method of the present disclosure are characterized by applying a CNN-based filter to a first picture and at least one of a quantization parameter map and a block partition map to output a second picture.
1. A video decoding method using a convolutional neural network (CNN)-based filter, the method comprising:
obtaining an input data, the input data including pixel data of a first reconstructed picture region which is partitioned into a plurality of coding units, and a quantization parameter map and a block partition map which are associated with the first reconstructed picture region, the first reconstructed picture region having been reconstructed from a bitstream of a video data, wherein the quantization parameter map is a two dimensional array for representing quantization parameters for the respective coding units constituting the first reconstructed picture region, and the block partition map is a two dimensional array for representing boundaries between the coding units in the first reconstructed picture region; and
providing the neural network based filter with the input data to obtain a second picture region that the first reconstructed picture region is filtered with the neural network based filter, wherein the neural network based filter has filter coefficients which have been trained with training data including pixel data of sample picture regions, and quantization parameter maps and block partition maps associated with the sample picture regions.
2. The method of claim 1 , wherein the quantization parameter map is constructed at the same resolution as the first reconstructed picture region, and is filled with quantization parameters for the coding units constituting the first reconstructed picture region.
3. The method of claim 1 ,
wherein the input data includes a block mode map which indicates an encoding mode for each of the coding units constituting the first reconstructed picture region.
4. The method of claim 1 , wherein the block partition map represents pixels indicating boundary of the coding block and pixels indicating an inner region of the coding block as different values.
5. The method of claim 4 , wherein, in the block partition map, a number of pixels indicating the boundary of the coding block is depending on at least one of a size of the coding block, a value of a quantization parameter, an encoding mode, a number of pixels to be updated, and a number of pixels to be referred to for filtering.
6. The method of claim 4 , wherein, in the block partition map, the pixels indicating the boundary of the coding block have different values depending on at least one of a size of the coding block, a value of a quantization parameter, a coding mode, a number of pixels to be updated, and a number of pixels to be referred to for filtering.
7. The method of claim 1 , wherein the filter coefficients of the neural network based filter are received from a video encoding apparatus.
8. A video decoding apparatus using a neural network based filter, the apparatus comprising:
an input unit configured to receive an input data, the input data including pixel data of a first reconstructed picture region, and a quantization parameter map and a block partition map which are associated with the first reconstructed picture region, the first reconstructed picture region having been reconstructed from a bitstream of a video data, wherein the quantization parameter map is a two dimensional array for representing quantization parameters for the respective coding units constituting the first reconstructed picture region, and the block partition map is a two dimensional array for representing boundaries between the coding units in the first reconstructed picture region;
a filter unit configured to apply the neural network based filter to the input data; and
an output unit configured to output a second picture region obtained from an output of the neural network based filter, the second picture region being a filtered picture region of the first reconstructed picture region, wherein the neural network based filter has filter coefficients which have been trained with training data including pixel data of sample picture regions, and quantization parameter maps and block partition maps associated with the sample picture regions.
9. The apparatus of claim 8 , wherein the quantization parameter map is constructed at the same resolution as the first reconstructed picture, and is filled with quantization parameters for the coding units constituting the first reconstructed picture region.
10. The apparatus of claim 8 , wherein the input data includes a block mode map which indicates an encoding mode for each of the coding units constituting the first reconstructed picture region.
11. The apparatus of claim 10 , wherein the training data includes block mode maps associated with the sample picture regions.
12. The apparatus of claim 8 , wherein the block partition map represents pixels indicating a boundary of a coding block and pixels indicating an inner region of the coding block with different values.
13. The apparatus of claim 12 , wherein, in the block partition map, a number of pixels indicating the boundary of the coding block is depending on at least one of a size of the coding block, a value of a quantization parameter, an encoding mode, a number of pixels to be updated, and a number of pixels to be referred to for filtering.
14. The apparatus of claim 12 , wherein, in the block partition map, the pixels indicating the boundary of the coding block have different values depending on at least one of a size of the coding block, a value of a quantization parameter, a coding mode, a number of pixels to be updated, and a number of pixels to be referred to for filtering.
15. The apparatus of claim 8 , wherein the filter coefficients of the neural network based filter are received from a video encoding apparatus.