Apparatus and method for image encoding and decoding
There is provide an image encoding method including transforming an image block into a first latent representation based on an invertible neural network, transforming the first latent representation into a second latent representation based on a non-invertible neural network, estimating a probability distribution of the first latent representation, and performing entropy encoding on the first latent representation based on the probability distribution by using an entropy encoder.
1 . An image encoding method comprising:
transforming an image block into a first latent representation based on an invertible neural network;
transforming, by a hyperprior encoder, the first latent representation into a second latent representation in units of sub-blocks based on a non-invertible neural network;
estimating a first probability distribution of the first latent representation based on the second latent representation;
performing entropy encoding on the first latent representation based on the first probability distribution by using a first entropy encoder; and
performing entropy encoding on the second latent representation based on a second probability distribution by using a second entropy encoder,
wherein the estimating of the first probability distribution comprises:
entropy decoding of the second latent representation by using a hyperprior decoder,
obtaining a hyperprior based on a result of the entropy decoding of the second latent representation; and
estimating the first probability distribution of the first latent representation based on the hyperprior by using a context estimator.
2 . The method of claim 1 , wherein the invertible neural network comprises a normalizing-flow neural network comprising one or more coupling layers.
3 . The method of claim 2 , wherein a parameter of the one or more coupling layers are obtained through training based on at least one of a neural network structure or data distribution.
4 . The method of claim 1 , wherein the estimating of the first probability distribution comprises:
dividing the first latent representation into a plurality of groups, and
estimating the first probability distribution for each of the plurality of groups.
5 . The method of claim 1 , wherein the estimating of the first probability distribution comprises obtaining a hyperprior as a probability distribution of the first latent representation by using a hyperprior decoder.
6 . The method of claim 1 , further comprising:
dividing the image block into a plurality of sub-blocks.
7 . The method of claim 6 , further comprising:
performing an operation within each of the plurality of sub-blocks based on one or more coupling layers of a normalizing-flow neural network.
8 . The method of claim 6 , further comprising:
performing an operation between the plurality of sub-blocks based on one or more coupling layers of a normalizing-flow neural network.
9 . The method of claim 8 , wherein the operation between the plurality of sub-blocks is performed by at least one of a hyperprior encoder, a hyperprior decoder, or a context estimator.
10 . The method of claim 1 , wherein the transforming of the image block into the first latent representation comprises transforming the image block into the first latent representation by hierarchically using two or more first modules.
11 . The method of claim 10 , wherein the transforming of the image block into the first latent representation comprises inputting a portion of a third latent representation, which is transformed by a first module of a previous layer, to a first module of a next layer to transform the portion of the third latent representation into a fourth latent representation, and combining a remaining portion of the third latent representation with the fourth latent representation to transform the combined latent representation into the first latent representation.
12 . An image decoding method comprising:
receiving a first bitstream of a first latent representation, obtained by transforming an image block based on an invertible neural network;
receiving a second bitstream of a second latent representation obtained by transforming, by a hyperprior encoder, the first latent representation in units of sub-blocks based on a non-invertible neural network;
estimating a first probability distribution of the first latent representation based on the second bitstream; and
reconstructing the image block based on the first bitstream and the first probability distribution,
wherein the estimating of the first probability distribution comprises:
entropy decoding of the second latent representation by using a hyperprior decoder,
obtaining a hyperprior based on a result of the entropy decoding of the second latent representation; and
estimating the first probability distribution of the first latent representation based on the hyperprior by using a context estimator.
13 . The method of claim 12 , further comprising:
entropy decoding the first bitstream based on the first probability distribution of the first latent representation by using a first entropy decoder,
wherein the reconstructing of the image block comprises reconstructing the image block based on a result of the entropy decoding of the first bitstream and the first probability distribution.
14 . The method of claim 12 , further comprising:
entropy decoding the second bitstream based on a second probability distribution by using a second entropy decoder.
15 . The method of claim 14 , wherein the estimating of the first probability distribution of the first latent representation comprises obtaining a hyperprior based on a result of decoding of the second bitstream, and estimating the first probability distribution of the first latent representation based on the hyperprior.
16 . An electronic device comprising:
a memory storing one or more instructions, and
a processor configured to execute the one or more instructions to implement:
an invertible neural network configured to transform an image block into a first latent representation;
a non-invertible neural network configured to:
transform, by a hyperprior encoder, the first latent representation into a second latent representation in units of sub-blocks, and
estimate a first probability distribution of the first latent representation based on the second latent representation; and
an entropy encoder configured to:
perform entropy encoding on the first latent representation based on the first probability distribution, and
perform entropy encoding on the second latent representation based on a second probability distribution by using a second entropy encoder,
wherein the estimating of the first probability distribution comprises:
entropy decoding of the second latent representation by using a hyperprior decoder,
obtaining a hyperprior based on a result of the entropy decoding of the second latent representation; and
estimating the first probability distribution of the first latent representation based on the hyperprior by using a context estimator.
17 . An electronic device comprising:
a memory storing one or more instructions, and
a processor configured to execute the one or more instructions to implement:
a non-invertible neural network configured to:
receive first bitstream of a first latent representation obtained by transforming an image block, and a second bitstream of a second latent representation obtained by transforming, by hyperprior encoder, the first latent representation in units of sub-blocks, and
estimate a probability distribution of the first latent representation; and
an invertible neural network configured to reconstruct an image based on the first bitstream and the estimated probability distribution,
wherein the estimating of the first probability distribution comprises:
entropy decoding of the second latent representation by using a hyperprior decoder,
obtaining a hyperprior based on a result of the entropy decoding of the second latent representation; and
estimating the first probability distribution of the first latent representation based on the hyperprior by using a context estimator.