Image compression and decoding, video compression and decoding: methods and systems
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
1. A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of:
(i) receiving an input image at a first computer system;
(ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation;
(iii) quantizing the latent representation using the first computer system to produce a quantized latent;
(iv) entropy encoding the quantized latent into a bitstream, using the first computer system;
(v) transmitting the bitstream to a second computer system;
(vi) the second computer system entropy decoding the bitstream to produce the quantized latent;
(vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image,
wherein in step (iv) a predefined probability distribution is used for the entropy encoding and wherein in step (vi) the predefined probability distribution is used for the entropy decoding; and
wherein the latent representation is defined in a latent space, and wherein the latent space is partitioned into chunks having intervariable correlations, and wherein chunks that are above a first distance threshold apart from each other and have no mutual influence have zero correlation, and wherein the number of parameters of the predefined probability distribution is determined by the partition size.
2. The method of claim 1 , wherein in step (iv) parameters characterizing a probability distribution are calculated, wherein a probability distribution characterised by the parameters is used for the entropy encoding, and wherein in step (iv) the parameters characterizing the probability distribution are included in the bitstream, and wherein in step (vi) the probability distribution characterised by the parameters is used for the entropy decoding.
3. The method of claim 1 , wherein the probability distribution is a factorized probability distribution.
4. The method of claim 3 , wherein the factorized probability distribution is a factorized normal distribution, and wherein the obtained probability distribution parameters are a respective mean and standard deviation of each respective element of the quantized latent.
5. The method of claim 3 , wherein the factorized probability distribution is a parametric factorized probability distribution.
6. The method of claim 5 , wherein the parametric factorized probability distribution is a continuous parametric factorized probability distribution.
7. The method of claim 5 , wherein the parametric factorized probability distribution is a discrete parametric factorized probability distribution.
8. The method of claim 5 , wherein parameters included in the parametric factorized probability distribution include shape, asymmetry, skewness and/or higher moment parameters.
9. The method of claim 5 , wherein the parametric factorized probability distribution is a parametric multivariate distribution.
10. The method of claim 1 , wherein the chunks have different sizes, shapes and extents.
11. The method of claim 1 , wherein a covariance matrix is used to characterise the parametrisation of intervariable dependences.
12. The method of claim 1 , wherein the predefined probability distribution is a continuous probability distribution with a well-defined probability density function (PDF), but lacking a well-defined or tractable formulation of its cumulative density function (CDF), and wherein numerical integration is used through Monte Carlo (MC) or Quasi-Monte Carlo (QMC) based methods.
13. The method of claim 1 , wherein the predefined probability distribution is defined by a multivariate cumulative distribution function, and wherein a copula is used as said multivariate cumulative distribution function.
14. The method of claim 1 , comprising obtaining a probability density function over a latent space of the latent representation by transforming a corresponding characteristic function using a Fourier Transform to obtain the probability density function.
15. The method of claim 1 , wherein to evaluate joint probability distributions over a pixel space of the input image, latent space is transformed into characteristic function space, a characteristic function is evaluated, and the output is converted back into the joint probability distribution space to evaluate said joint probability distribution over the pixel space of the input image.
16. The method of claim 1 , wherein said predefined probability distribution is defined by a mixture model.
17. The method of claim 16 , wherein the mixture model comprises a weighted sum of one or more base distributions as mixture components, the one or more base distributions comprising parametric or non-parametric, factorized or non-factorisable multivariate distributions.
18. The method of claim 1 , wherein a factorized probability distribution is a non-parametric factorized probability distribution.
19. The method of claim 1 , wherein the predefined probability distribution is a non-factorisable parametric multivariate distribution.
20. The method of claim 1 , wherein the partitioned latent space is partitioned based on spatial dimensions of the latent representation.
21. The method of claim 1 , wherein the partitioned latent space is partitioned based on channel dimensions of the latent representation.
22. The method of claim 1 , wherein the partitioned latent space is partitioned into chunks of different sizes, shapes or extents.
23. The method of claim 1 , wherein one or more partitions of the partitioned latent space are correlated.
24. The method of claim 1 , wherein one or more partitions of the partitioned latent space have zero correlation.