Granted Patent B2

US 12,278,957 · App. 18/127,616 · Granted Apr 15, 2025

Inventors: Yanzhuo Ma (Dongguan, CN); Ruipeng Qiu (Dongguan, CN); Junyan Huo (Dongguan, CN); Shuai Wan (Dongguan, CN); Fuzheng Yang (Dongguan, CN)

Assignee: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.

H04N19/117G06V10/82H04N19/17H04N19/184

Abstract

Claims (52)

1. A method for encoding a video, applicable to an encoder, comprises:

determining side information having a correlation with a current coding unit;

filtering first relevant information to be coded of the current coding unit by using a preset network model and the side information, to obtain second relevant information to be coded; and

inputting the second relevant information to be coded into a subsequent coding module for encoding, to obtain a bitstream;

wherein the preset network model comprises a first network model, and the first network model comprises at least a first neural network model and a first adder; and

the first neural network model comprises: a first convolution layer, at least one second convolution layer, first residual layers, second residual layers, average pooling layers, and sampling rate conversion modules; wherein

the first convolution layer is followed by at least two of the first residual layers connected in series, and an average pooling layer is connected between adjacent first residual layers;

at least one of the first residual layers is followed by at least two of the second residual layers connected in series, and a sampling rate conversion module is connected between adjacent second residual layers; and

at least one of the second residual layers is connected in series with the at least one second convolution layer;

wherein filtering the first relevant information to be coded of the current coding unit by using the preset network model and the side information, to obtain the second relevant information to be coded comprises:

obtaining the first decoding information corresponding to first information to be coded of the current coding unit; and

inputting the first decoding information and the side information into the first network model to output the second decoding information, wherein the first network model is used for performing enhancement on the first decoding information by using the side information;

wherein part of input ends of the first network model is shorted to an output end of the first network model, and an input end for the first decoding information in the first network model is shorted to the output end.

2. The method of claim 1 , wherein the coding unit is a picture or an area in the picture.

3. The method of claim 1 , wherein the first relevant information to be coded comprises first decoding information, and the second relevant information to be coded comprises second decoding information.

4. The method of claim 3 , wherein

inputting the second relevant information to be coded into the subsequent coding module for encoding, to obtain the bitstream comprises:

reconstructing the current coding unit by using the second decoding information, to obtain a reconstructed unit of the current coding unit; and

performing subsequent coding according to the reconstructed unit of the current coding unit, to obtain the bitstream.

5. The method of claim 1 , wherein inputting the first decoding information and the side information into the first network model to output the second decoding information, comprises:

inputting the first decoding information and the side information into the first neural network model to output a first intermediate value; and

adding the first intermediate value to the first decoding information by the first adder, to obtain the second decoding information.

6. A method for decoding a video, applicable to a decoder, comprises:

decoding a bitstream to obtain information to be decoded;

inputting the information to be decoded into a first decoding module to output first decoding information of a current coding unit;

determining side information having a correlation with the first decoding information;

filtering the first decoding information by using a preset network model and the side information, to obtain second decoding information; and

reconstructing the current coding unit by using the second decoding information, to obtain a reconstructed unit of the current coding unit;

wherein the preset network model comprises a first network model, and the first network model comprises at least a first neural network model and a first adder; and

the first neural network model comprises: a first convolution layer, at least one second convolution layer, first residual layers, second residual layers, average pooling layers, and sampling rate conversion modules; wherein

the first convolution layer is followed by at least two of the first residual layers connected in series, and an average pooling layer is connected between adjacent first residual layers;

at least one of the first residual layers is followed by at least two of the second residual layers connected in series, and a sampling rate conversion module is connected between adjacent second residual layers; and

at least one of the second residual layers is connected in series with the at least one second convolution layer;

wherein filtering the first decoding information by using the preset network model and the side information, to obtain the second decoding information comprises:

inputting the first decoding information and the side information into the first network model to output the second decoding information;

wherein the first network model is used for performing filtering on the first decoding information according to the correlation between the first decoding information and the side information, part of input ends of the first network model is shorted to an output end of the first network model, and an input end for the first decoding information in the first network model is shorted to the output end.

7. The method of claim 6 , wherein the coding unit is a picture or an area in the picture.

8. The method of claim 6 , wherein inputting the first decoding information and the side information into the first network model to output the second decoding information comprises:

inputting the first decoding information and the side information into the first neural network model to output a first intermediate value; and

adding the first intermediate value to the first decoding information by the first adder, to obtain the second decoding information.

9. A decoder, comprises: a processor and a memory for storing a computer program executable by the processor, wherein the processor is configured to execute the computer program to:

decode a bitstream to obtain information to be decoded, and input the information to be decoded into a first decoding module to output first decoding information of a current coding unit;

determine side information having a correlation with the first decoding information;

filter the first decoding information by using a preset network model and the side information, to obtain second decoding information; and

reconstruct the current coding unit by using the second decoding information, to obtain a reconstructed unit of the current coding unit;

wherein the preset network model comprises a first network model, and the first network model comprises at least a first neural network model and a first adder; and

the first neural network model comprises: a first convolution layer, at least one second convolution layer, first residual layers, second residual layers, average pooling layers, and sampling rate conversion modules; wherein

the first convolution layer is followed by at least two of the first residual layers connected in series, and an average pooling layer is connected between adjacent first residual layers;

at least one of the first residual layers is followed by at least two of the second residual layers connected in series, and a sampling rate conversion module is connected between adjacent second residual layers; and

at least one of the second residual layers is connected in series with the at least one second convolution layer;

wherein the processor is further configured to execute the computer program to input the first decoding information and the side information into the first network model to output the second decoding information;

wherein the first network model is used for performing filtering on the first decoding information according to the correlation between the first decoding information and the side information, part of input ends of the first network model is shorted to an output end of the first network model, and an input end for the first decoding information in the first network model is shorted to the output end.

Assignments (1)

Continuity (2)

References Cited (32)