Method, apparatus, and storage medium using padding/trimming in compression neural network
An encoding apparatus extracts features of an image by applying multiple padding operations and multiple downscaling operations to an image represented by data and transmits feature information indicating the features to a decoding apparatus. The multiple padding operations and the multiple downscaling operations are applied to the image in an order in which one padding operation is applied and thereafter one downscaling operation corresponding to the padding operation is applied. A decoding method receives feature information from an encoding apparatus, and generates a reconstructed image by applying multiple upscaling operations and multiple trimming operations to an image represented by the feature information. The multiple upscaling operations and the multiple trimming operations are applied to the image in an order in which one upscaling operation is applied and thereafter one trimming operation corresponding to the upscaling operation is applied.
1. A decoding method performed by a decoding apparatus, comprising:
receiving feature information from an encoding apparatus; and
generating a reconstructed image by applying multiple upscaling operations and multiple trimming operations to an image represented by the feature information,
wherein each of the multiple trimming operations is configured to remove one or more lines, identical to a number of one or more lines that are added through a padding operation performed by the encoding apparatus, from the image to which the upscaling operation is applied.
2. The decoding method of claim 1 , wherein the multiple upscaling operations and the multiple trimming operations are applied to the image in an order in which one upscaling operation is applied to the image and thereafter one trimming operation corresponding to the upscaling operation is applied to the image.
3. The decoding method of claim 1 , wherein a number of pairs of the multiple upscaling operations and the multiple trimming operations is identical to a number of pairs of multiple padding operations and multiple downscaling operations performed by the encoding apparatus.
4. The decoding method of claim 1 , wherein each of the multiple upscaling operations is configured to adjust a size of the image to n times, where n is an integer equal to or greater than 2.
5. The decoding method of claim 1 , wherein:
each of the multiple upscaling operations is configured to increase a size of the image to n times, and
n is an integer equal to or greater than 2.
6. The decoding method of claim 1 , wherein number-of-lines information indicating numbers of one or more lines that are added through multiple padding operations performed by the encoding apparatus is received from the encoding apparatus.
7. The decoding method of claim 1 , wherein:
ratio information is received from the encoding apparatus,
the ratio information includes ratios of multiple downscaling operations performed by the encoding apparatus or reciprocals of the ratios of the multiple downscaling operations performed by the encoding apparatus.
8. The decoding method of claim 7 , wherein:
a k-th upscaling operation, among the multiple upscaling operations, is configured to adjust a size of the image to S k times,
k is an integer that is equal or greater than 1 and less than or equal to m,
m is a number of the multiple downscaling operations, and
S k is determined based on the ratio information.
9. The decoding method of claim 1 , wherein:
a k-th trimming operation on the image, among the multiple trimming operations, is configured to remove, from the image, a number of lines identical to a number of lines that are added through a k-th padding operation corresponding to the k-th trimming operation, among multiple padding operations performed by the encoding apparatus,
k is an integer that is equal to or greater than 1 and less than or equal to m, and
m is a number of the multiple downscaling operations.
10. The decoding method of claim 9 , wherein:
a padding operation performed in a k-th order, among m padding operations performed by the encoding apparatus, corresponds to a trimming operation performed in an (m−k+1)-th order, among m trimming operations performed by the decoding apparatus,
k is an integer that is equal to or greater than 1 and less than or equal to m, and
m is a number of the multiple downscaling operations.
11. A computer-readable storage medium storing a bitstream, the bitstream comprising feature information, wherein:
a reconstructed image is generated by applying multiple upscaling operations and multiple trimming operations to an image represented by the feature information,
wherein each of the multiple trimming operations is configured to remove one or more lines, identical to a number of one or more lines that are added through a padding operation performed by the encoding apparatus, from the image to which the upscaling operation is applied.