Image encoding and decoding method and apparatus
This application provides an image encoding and decoding method and apparatus. The image encoding method in this application includes: obtaining a to-be-processed first image feature; performing non-linear transformation processing on the to-be-processed first image feature to obtain a processed image feature, where the non-linear transformation processing sequentially includes a first non-linear operation, convolution processing, and an element-wise multiplication operation; and performing encoding based on the processed image feature to obtain a bitstream. This application can avoid limitation on a convolutional parameter, to implement efficient non-linear transformation processing in an encoding/decoding network, and further improve rate-distortion performance of an image/video compression algorithm.
1 . An image encoding method, comprising:
obtaining a to-be-processed first image feature;
performing non-linear transformation processing on the to-be-processed first image feature to obtain a processed image feature, wherein the non-linear transformation processing sequentially comprises a first non-linear operation, convolution processing, and an element-wise multiplication operation; and
performing encoding based on the processed image feature to obtain a bitstream,
wherein the performing non-linear transformation processing on the to-be-processed first image feature to obtain a processed image feature comprises;
performing the first non-linear operation on each feature value in the to-be-processed first image feature to obtain a second image feature;
performing the convolution processing on the second image feature to obtain a third image feature, wherein a plurality of feature values in the third image feature correspond to a plurality of feature values in the to-be-processed first image feature; and
performing the element-wise multiplication operation on the plurality of corresponding feature values in the to-be-processed first image feature and the third image feature to obtain the processed image feature.
2 . The method according to claim 1 , wherein the first non-linear operation comprises an activation function of rectified linear unit series, Sigmoid, Tanh, or piecewise linear mapping.
3 . The method according to claim 1 , further comprising:
constructing a non-linear transformation unit in a training phase, wherein the non-linear transformation unit in the training phase comprises a first non-linear operation layer, a convolution processing layer, and an element-wise multiplication operation layer; and
performing training based on pre-obtained training data to obtain a trained non-linear transformation unit for implementing the non-linear transformation processing.
4 . An image encoding device, comprising:
one or more processors; and
a non-transitory computer-readable storage medium, coupled to the one or more processors and storing instructions, which, when executed by the one or more processors, cause the image encoding device to perform operations comprising:
obtaining a to-be-processed first image feature;
performing non-linear transformation processing on the to-be-processed first image feature to obtain a processed image feature, wherein the non-linear transformation processing sequentially comprises a first non-linear operation, convolution processing, and an element-wise multiplication operation; and
performing encoding based on the processed image feature to obtain a bitstream,
wherein the performing non-linear transformation processing on the to-be-processed first image feature to obtain processed image feature comprises:
performing the first non-linear operation on each feature value in the to-be-processed first image feature to obtain a second image feature;
performing the convolution processing on the second image feature to obtain a third image feature, wherein a plurality of features values in the third image feature correspond to a plurality of feature values in the to-be-processed first image feature, and
performing the element-wise multiplication operation on the plurality of corresponding feature values in the to-be-processed first image feature and the third image feature to obtain the processed image feature.
5 . The device according to claim 4 , wherein the first non-linear operation comprises an activation function of rectified linear unit series, Sigmoid, Tanh, or piecewise linear mapping.
6 . The device according to claim 4 , wherein the operations further comprise:
constructing a non-linear transformation unit in a training phase, wherein the non-linear transformation unit in the training phase comprises a first non-linear operation layer, a convolution processing layer, and an element-wise multiplication operation layer; and
performing training based on pre-obtained training data to obtain a trained non-linear transformation unit for implementing the non-linear transformation processing.
7 . A non-transitory computer-readable storage medium, comprising instructions, wherein when the instructions are run on a computer, cause the computer to perform operations comprising:
obtaining a to-be-processed first image feature;
performing non-linear transformation processing on the to-be-processed first image feature to obtain a processed image feature, wherein the non-linear transformation processing sequentially comprises a first non-linear operation, convolution processing, and an element-wise multiplication operation; and
performing encoding based on the processed image feature to obtain a bitstream,
wherein the performing non-linear transformation processing on the to-be-processed first image feature to obtain a processed image feature comprises:
performing the first non-linear operation on each feature value in the to-be processed first image feature to obtain a second image feature;
performing the convolution processing on the second image feature to obtain a third image feature, wherein a plurality of feature values in the third image feature correspond to a plurality of feature values in the to-be-processed first image feature; and
performing the element-wise multiplication operation on the plurality of corresponding feature values in the to-be-processed first image feature and the third image feature to obtain the processed image feature.
8 . The non-transitory computer-readable storage medium according to claim 7 , wherein the first non-linear operation comprises an activation function of rectified linear unit series, Sigmoid, Tanh, or piecewise linear mapping.
9 . The non-transitory computer-readable storage medium according to claim 7 , wherein the operations further comprise:
constructing a non-linear transformation unit in a training phase, wherein the non-linear transformation unit in the training phase comprises a first non-linear operation layer, a convolution processing layer, and an element-wise multiplication operation layer; and
performing training based on pre-obtained training data to obtain a trained non-linear transformation unit for implementing the non-linear transformation processing.