Granted Patent B2

US 12,483,733 · App. 18/191,990 · Granted Nov 25, 2025

Inventors: Jue Mao (Hangzhou, CN); Haitao Yang (Shenzhen, CN); Xiang Ma (Moscow, RU)

Assignee: HUAWEI TECHNOLOGIES CO., LTD.

H04N19/91H04N19/132H04N19/172H04N19/176H04N19/42H04N19/50H04N19/70

Abstract

Claims (127)

1 . An entropy encoding method, comprising:

obtaining base layer information of a to-be-encoded picture block, wherein

the base layer information corresponds to M samples in the to-be-encoded picture block, and

M is a positive integer;

obtaining K elements corresponding to enhancement layer information of the to-be-encoded picture block, wherein

the enhancement layer information corresponds to N samples in the to-be-encoded picture block,

both K and N are positive integers, and

N≥M;

inputting the base layer information into a neural network to obtain K groups of probability values, wherein

the K groups of probability values correspond to the K elements, and

any group of probability values, from the K groups of probability values, represents probabilities of a plurality of candidate values of a corresponding element from the K elements; and

performing entropy encoding, on the K elements based on the K groups of probability values, to generate an encoded bit stream, wherein obtaining the K elements corresponding to the enhancement layer information of the to-be-encoded picture block comprises at least:

performing hybrid encoding on the original values of the N samples to obtain encoded values of the N samples;

performing hybrid decoding on the encoded values of the N samples to obtain reconstruction values of the N samples;

performing difference calculation based on the original values of the N samples and the reconstruction values of the N samples to obtain first differences of the N samples, and

performing feature extraction on the first differences of the N samples to obtain K difference eigenvalues, wherein the K elements are the K difference eigenvalues.

2 . The entropy encoding method of claim 1 , wherein

the base layer information comprises reconstruction values of the M samples,

the base layer information comprises predicted values and/or residual values of the M samples, or

the base layer information comprises transform values of the M samples.

3 . The entropy encoding method of claim 1 , wherein obtaining the K elements corresponding to the enhancement layer information of the to-be-encoded picture block further comprises at least one of:

i) performing feature extraction on original values of the N samples to obtain K eigenvalues, wherein

the K elements are the K eigenvalues;

ii) obtaining predicted values of the N samples;

performing a difference calculation based on the original values of the N samples and the predicted values of the N samples to obtain second differences of the N samples;

transforming the second differences of the N samples to obtain transform values of the N samples; and

performing feature extraction on the transform values of the N samples to obtain K transform value eigenvalues, wherein

the K elements are the K transform value eigenvalues; or

iii) obtaining predicted values of the N samples;

performing a difference calculation based on the original values of the N samples and the predicted values of the N samples to obtain second differences of the N samples;

transforming the second differences of the N samples to obtain transform values of the N samples;

quantizing the transform values of the N samples to obtain quantized transform values of the N samples; and

extracting coordinates of a last non-zero value from the quantized transform values of the N samples, a first value of the quantized transform values of the N samples, and all values between a second value of the quantized transform values of the N samples and the last non-zero value, wherein

the K elements comprise the coordinates of the last non-zero value, the first value, and the all values.

4 . The entropy encoding method of claim 3 , wherein obtaining the predicted values of the N samples comprises:

performing hybrid encoding on original values of the M samples to obtain encoded values of the M samples;

performing hybrid decoding on the encoded values of the M samples to obtain the reconstruction values of the M samples; and

obtaining the predicted values of the N samples based on the reconstruction values of the M samples.

5 . The entropy encoding method of claim 4 , wherein obtaining the predicted values of the N samples based on the reconstruction values of the M samples comprises:

when N=M, obtaining the predicted values of the N samples based on the reconstruction values of the M samples, or

when N>M, obtaining reconstruction values of the N samples by performing interpolation on the reconstruction values of the M samples, and obtaining the predicted values of the N samples based on the reconstruction values of the N samples.

6 . The entropy encoding method of claim 3 , wherein

when probabilities of a plurality of candidate values of a first element satisfy a Gaussian distribution, a group of probability values corresponding to the first element comprises a variance and a mean of the Gaussian distribution satisfied by the probabilities of the plurality of candidate values of the first element, and

the first element is any one of the K elements.

7 . The entropy encoding method of claim 1 , wherein

the neural network comprises a convolution layer,

a depth of a convolution kernel of the convolution layer is 24, 32, 48, 64, 192, or 384, and

a size of the convolution kernel of the convolution layer is 1×1, 3×3, 5×5, or 7×7.

8 . The entropy encoding method of claim 7 , further comprising:

obtaining a training dataset, wherein (a) the training dataset comprises original values of M samples of each of a plurality of picture blocks and K groups of probability values respectively corresponding to N samples of each of the plurality of picture blocks, or (b) the training dataset comprises quantized transform values of the M samples of each of the plurality of picture blocks and the K groups of probability values respectively corresponding to the N samples of each of the plurality of picture blocks; and

obtaining the neural network by performing training based on the training dataset.

9 . An entropy decoding method, comprising:

obtaining a bit stream;

obtaining base layer information and encoded data of a to-be-decoded picture block by parsing the bit stream, wherein

the base layer information corresponds to M samples in the to-be-decoded picture block,

the encoded data corresponds to enhancement layer information,

the enhancement layer information corresponds to N samples in the to-be-decoded picture block,

both M and N are positive integers, and

N≥M;

inputting the base layer information into a neural network to obtain K groups of probability values, wherein

the K groups of probability values correspond to K elements,

any group of probability values, from the K groups of probability values, represents probabilities of a plurality of candidate values of a corresponding element from the K elements, and

K is a positive integer;

obtaining the K elements by performing entropy decoding on the encoded data based on the K groups of probability values; and

obtaining reconstruction values of the N samples based on the K elements, wherein N is a positive integer, wherein obtaining the reconstruction values of the N samples based on the K elements comprises at least:

in association with the K elements being K difference eigenvalues, obtaining first differences of the N samples by performing inverse feature extraction on the K difference eigenvalues;

parsing the bit stream to obtain reconstruction values of the M samples corresponding to the base layer information of the to-be-decoded picture block, wherein N≥M; and

obtaining the reconstruction values of the N samples based on the reconstruction values of the M samples and the first differences of the N samples.

10 . The entropy decoding method of claim 9 , wherein obtaining the reconstruction values of the N samples based on the K elements further comprises at least one of:

i) when the K elements are K eigenvalues, obtaining the reconstruction values of the N samples by performing inverse feature extraction on the K eigenvalues;

ii) when the K elements are K transform value eigenvalues, obtaining transform values of the N samples by performing inverse feature extraction on the K transform value eigenvalues;

inversely transforming the transform values of the N samples to obtain second differences of the N samples;

parsing the bit stream to obtain reconstruction values of the M samples corresponding to the base layer information of the to-be-decoded picture block, wherein N≥M;

obtaining predicted values of the M samples based on the reconstruction values of the M samples; and

obtaining the reconstruction values of the N samples based on the predicted values of the M samples and the second differences of the N samples; or

iii) when the K elements comprise coordinates of a last non-zero value of transform values of the N samples, a first value of the transform values of the N samples, and all values between a second value of the transform values of the N samples and the last non-zero value, performing inverse quantization based on the coordinates of the last non-zero value, the first value, and the all values to obtain the transform values of the N samples;

inversely transforming the transform values of the N samples to obtain second differences of the N samples;

parsing the bit stream to obtain reconstruction values of the M samples corresponding to the base layer information of the to-be-decoded picture block, wherein N≥M;

obtaining predicted values of the M samples based on the reconstruction values of the M samples; and

obtaining the reconstruction values of the N samples based on the predicted values of the M samples and the second differences of the N samples.

11 . The entropy decoding method of claim 10 , wherein obtaining the reconstruction values of the N samples based on the reconstruction values of the M samples and the first differences of the N samples comprises:

when N>M, obtaining reconstruction values of the N samples by performing interpolation on the reconstruction values of the M samples, and obtaining final reconstruction values of the N samples by performing a sum calculation based on the reconstruction values of the N samples and the first differences of the N samples; or

when N=M, obtaining reconstruction values of the N samples by performing the sum calculation based on the reconstruction values of the M samples and the first differences of the N samples.

12 . The entropy decoding method of claim 10 , wherein obtaining the reconstruction values of the N samples based on the predicted values of the M samples and the second differences of the N samples comprises:

when N>M, obtaining predicted values of the N samples by performing interpolation on the predicted values of the M samples, and obtaining the reconstruction values of the N samples by performing a sum calculation based on the predicted values of the N samples and the second differences of the N samples; or

when N=M, obtaining reconstruction values of the N samples by performing a sum calculation based on the predicted values of the M samples and the second differences of the N samples.

13 . A decoder, comprising:

a processor; and

a memory operatively coupled to the processor and configured to store a program to be executed by the processor, wherein the program, when executed by the processor, causes the decoder to:

obtain a bit stream;

obtain base layer information and encoded data of a to-be-decoded picture block by parsing the bit stream, wherein

the base layer information corresponds to M samples in the to-be-decoded picture block,

the encoded data corresponds to enhancement layer information,

the enhancement layer information corresponds to N samples in the to-be-decoded picture block,

both M and N are positive integers, and

N≥M;

input the base layer information into a neural network to obtain K groups of probability values, wherein

the K groups of probability values correspond to K elements,

any group of probability values, from the K groups of probability values, represents probabilities of a plurality of candidate values of a corresponding element from the K elements, and

K is a positive integer;

obtain the K elements by performing entropy decoding on the encoded data based on the K groups of probability values; and

obtain reconstruction values of the N samples based on the K elements, wherein N is a positive integer, wherein the decoder obtains the reconstruction values of the N samples according to at least:

when the K elements are K difference eigenvalues, obtain first differences of the N samples by performing inverse feature extraction on the K difference eigenvalues,

parse the bit stream to obtain reconstruction values of the M samples corresponding to the base layer information of the to-be-decoded picture block, wherein N≥M, and

obtain the reconstruction values of the N samples based on the reconstruction values of the M samples and the first differences of the N samples.

14 . The decoder of claim 13 , wherein the decoder further obtains the reconstruction values of the N samples according to at least one of:

i) when the K elements are K eigenvalues, obtain the reconstruction values of the N samples by performing inverse feature extraction on the K eigenvalues;

ii) when the K elements are K transform value eigenvalues, obtain transform values of the N samples by performing inverse feature extraction on the K transform value eigenvalues;

inversely transform the transform values of the N samples to obtain second differences of the N samples;

parse the bit stream to obtain reconstruction values of the M samples corresponding to the base layer information of the to-be-decoded picture block, wherein N≥M;

obtain predicted values of the M samples based on the reconstruction values of the M samples; and

obtain the reconstruction values of the N samples based on the predicted values of the M samples and the second differences of the N samples; or

iii) when the K elements comprise coordinates of a last non-zero value of transform values of the N samples, a first value of the transform values of the N samples, and all values between a second value of the transform values of the N samples and the last non-zero value, perform inverse quantization based on the coordinates of the last non-zero value, the first value, and the all values to obtain the transform values of the N samples;

inversely transform the transform values of the N samples to obtain second differences of the N samples;

parse the bit stream to obtain reconstruction values of the M samples corresponding to the base layer information of the to-be-decoded picture block, wherein N≥M;

obtain predicted values of the M samples based on the reconstruction values of the M samples; and

obtain the reconstruction values of the N samples based on the predicted values of the M samples and the second differences of the N samples.

15 . The decoder of claim 14 , wherein the decoder is further caused to:

when N>M, obtain reconstruction values of the N samples by performing interpolation on the reconstruction values of the M samples, and obtain final reconstruction values of the N samples by performing a sum calculation based on the reconstruction values of the N samples and the first differences of the N samples; or

when N=M, obtain reconstruction values of the N samples by performing a sum calculation based on the reconstruction values of the M samples and the first differences of the N samples.

16 . The decoder of claim 14 , wherein the decoder is further caused to:

when N>M, obtain predicted values of the N samples by performing interpolation on the predicted values of the M samples, and obtain the reconstruction values of the N samples by performing a sum calculation based on the predicted values of the N samples and the second differences of the N samples; or

when N=M, obtain reconstruction values of the N samples by performing a sum calculation based on the predicted values of the M samples and the second differences of the N samples.

17 . The entropy encoding method of claim 1 , wherein the neural network uses a variance and a mean of Gaussian distribution satisfied by each of the K elements to represent the probabilities of the plurality of candidate values.

18 . A non-transitory computer readable store medium configured to store computer readable instructions that, when executed by a processor of an information processing device, cause the information processing device to provide execution comprising the entropy encoding method of claim 1 .

19 . A non-transitory computer readable store medium configured to store computer readable instructions that, when executed by a processor of an information processing device, cause the information processing device to provide execution comprising the entropy decoding method of claim 9 .

20 . The entropy encoding method of claim 1 , wherein the encoded bit stream includes the base layer information and encoded data of a to-be-decoded picture block, and the encoded data corresponds to the enhancement layer information.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (16)