Method and apparatus for encoding a picture and decoding a bitstream using a neural network
A method and an apparatus for encoding a picture and decoding a bitstream representing a matrix using a neural network is provided. The method includes obtaining a resizing method out of a plurality of resizing methods. The resizing method is applied to resize an input of size S to a size S . The resized input is processed by the neural network, wherein the neural network comprises one or more downsampling layers. Subsequently, the method provides an output of the neural network, the output having a size P that is smaller than S in the at least one dimension.
1 . A method of decoding a bitstream representing a picture using a neural network (NN) to process an input representing a matrix having a size in at least one dimension, wherein the method comprises:
obtaining a resizing method out of a plurality of resizing methods, wherein obtaining the resizing method is based on one or more indications, wherein the one or more indications comprise a third indication, wherein the third indication has a value that indicates an interpolation filter that is to be used in the interpolation, wherein, the third indication is or comprises an index indicating an entry in a first lookup table (LUT) that has a plurality of entries and each entry in the first LUT specifies the interpolation filter, wherein obtaining the resizing method comprises comparing a larger size with another size and obtaining, based on the comparing, the resizing method, wherein the another size is obtained from a function comprising a combined upsampling parameter of the NN, the upsampling parameter being a product of upsampling ratios of all upsampling layers of the NN, wherein each upsampling layer has a different upsampling ratio;
processing the input with the size by the NN, wherein the NN comprises one or more upsampling layers, thereby obtaining an intermediate output having the larger size that is larger than the size in the at least one dimension; and
resizing the intermediate output from the larger size to the another size by applying the obtained resizing method, thereby obtaining a decoded picture.
2 . The method according to claim 1 , wherein the step of obtaining the resizing method comprises determining the resizing method from the plurality of resizing methods based on information relating to at least one of the input, the one or more layers of the NN, an output to be provided by the NN, or the decoded picture.
3 . The method according to claim 1 , wherein the plurality of resizing methods comprises padding, padding with zeros, reflection padding, repetition padding, cropping, interpolation to increase the larger size of the intermediate output to the another size, interpolation to decrease the larger size of the intermediate output to the another size.
4 . The method according to claim 1 , wherein the another size is further obtained from:
(ii) a product of the size and the unsampling ratios of all unsampling layers of the neural network; or
(iii) the bitstream or an additional bitstream or an index in the bitstream or an index in the additional bitstream and the index indicating an entry in a table, wherein the table is a lookup table (LUT), comprising a plurality of entries and each entry indicates the another size, wherein the method further comprises obtaining the another size using the index.
5 . The method according to claim 1 , wherein:
based on determining that the larger size is not equal to the another size, the resizing method is applied; or
based on determining that the larger size is smaller than the another size, the resizing method is applied that increases the larger size; or
based on determining that the larger size is larger than the another size, the resizing method is applied that decreases the larger size.
6 . The method according claim 2 , wherein the one or more indications comprise at least one of:
a first indication being or comprising a first flag having a size of 1 bit, wherein a first value of the first indication indicates that padding or cropping is to be applied as the resizing method and a second value of the first indication indicates that interpolation is to be applied as the resizing method;
a second indication, wherein the second indication has a first value that indicates that the larger size to be increased and a second value that indicates that the larger size is to be decreased;
a fourth indication, being or comprising a second flag having a size of 1 bit, the fourth indication having a first value that indicates that padding is to be applied as the resizing method and a second value that indicates that cropping is to be applied as the resizing method;
a fifth indication having a first value that indicates whether padding with zeros, reflection padding or repetition padding is to be applied as the resizing method; and
a sixth indication that is or comprises an index indicating an entry in a second look-up table (LUT) wherein the second LUT comprises a plurality of entries, wherein each entry specifies a resizing method.
7 . A decoder for decoding a bitstream representing a picture, wherein the decoder comprises one or more processors for implementing a neural network (NN) to process an input representing a matrix having a size in at least one dimension, wherein the one or more processors are adapted to perform a method comprising:
obtaining a resizing method out of a plurality of resizing methods, wherein obtaining the resizing method is based on one or more indications, wherein the one or more indications comprise a third indication, wherein the third indication has a value that indicates an interpolation filter that is to be used in the interpolation, wherein, the third indication is or comprises an index indicating an entry in a first lookup table (LUT) that has a plurality of entries and each entry in the first LUT specifies the interpolation filter, wherein obtaining the resizing method comprises comparing a larger size with another size and obtaining, based on the comparing, the resizing method, wherein the another size is obtained from a function comprising a combined upsampling parameter of the NN, the upsampling parameter being a product of upsampling ratios of all upsampling layers of the NN, wherein each upsampling layer has a different upsampling ratio;
processing the input with the size by the NN, wherein the NN comprises one or more upsampling layers, thereby obtaining an intermediate output having the larger size that is larger than the size in the at least one dimension; and
resizing the intermediate output from the larger size to the another size by applying the obtained resizing method, thereby obtaining a decoded picture.
8 . The method according to claim 1 , wherein the NN further comprises one or more downsampling layers configured to apply a downsampling by a certain factor.
9 . The method according to claim 8 , wherein the NN further comprises an inverse generalized divisive normalization (IGDN).
10 . The method according to claim 2 , wherein the upsampling parameter includes information regarding upsampling ratios of a decoder of the NN.