Data compression using conditional entropy models

1. A method implemented by a data processing apparatus, the method comprising:

processing data using an encoder neural network to generate a latent representation of the data;

processing the latent representation of the data using a hyper-encoder neural network to generate a latent representation of an entropy model, wherein the entropy model is defined by one or more probability distribution parameters characterizing one or more code symbol probability distributions;

generating an entropy encoded representation of the latent representation of the entropy model;

generating an entropy encoded representation of the latent representation of the data using the latent representation of the entropy model, comprising:

determining the probability distribution parameters defining the entropy model using the latent representation of the entropy model; and

generating the entropy encoded representation of the latent representation of the data using the entropy model; and

determining a compressed representation of the data from the entropy encoded representations of: (i) the latent representation of the data and (ii) the latent representation of the entropy model used to entropy encode the latent representation of the data.

2. The method of claim 1 , wherein the data comprises an image and the encoder neural network is a convolutional neural network.

3. The method of claim 1 , wherein processing the data using an encoder neural network to generate a latent representation of the data comprises:

determining an ordered collection of code symbols representing the data by quantizing the latent representation of the data.

4. The method of claim 1 , wherein generating the entropy encoded representation of the latent representation of the entropy model comprises:

quantizing the latent representation of the entropy model.

5. The method of claim 3 , wherein the entropy model is defined by a respective code symbol probability distribution for each code symbol in the ordered collection of code symbols representing the data.

6. The method of claim 1 , wherein each code symbol probability distribution is a Gaussian distribution convolved with a uniform distribution, and the respective probability distribution parameters defining each code symbol probability distribution comprise respective mean and standard deviation parameters of the Gaussian distribution.

7. The method of claim 4 , wherein generating the entropy encoded representation of the latent representation of the entropy model comprises:

entropy encoding the quantized latent representation of entropy model using one or more predetermined probability distributions.

8. The method of claim 3 , wherein generating the entropy encoded representation of the latent representation of the data using the entropy model comprises:

arithmetic encoding the ordered collection of code symbols representing the data using the code symbol probability distributions defining the entropy model.

9. The method of claim 1 , wherein determining the probability distribution parameters defining the entropy model using the latent representation of the entropy model comprises:

quantizing the latent representation of the entropy model; and

processing the quantized latent representation of the entropy model by one or more neural network layers.

10. The method of claim 9 , further comprising:

quantizing the latent representation of the data;

autoregressively processing code symbols of the quantized latent representation of the data by one or more neural network layers to generate a plurality of context outputs; and

processing the context outputs and the quantized latent representation of the entropy model by one or more neural network layers to generate the probability distribution parameters defining the entropy model.

11. The method of claim 10 , wherein:

the code symbols of the quantized latent representation of the data are associated with an ordering; and

autoregressively processing the code symbols of the quantized latent representation of the data by one or more neural network layers to generate a plurality of context outputs comprises:

generating a respective context output for each code symbol of the quantized latent representation of the data, wherein generating the context output for a given code symbol of the quantized latent representation of the data comprises:

processing an input that comprises one or more code symbols of the quantized latent representation of the data that precede the given code symbol of the quantized latent representation of the data using the one or more neural network layers to generate the context output for the given code symbol of the quantized latent representation of the data.

12. The method of claim 11 , wherein the input does not comprise either: (i) the given code symbol of the quantized latent representation of the data, or (ii) any code symbols of the quantized latent representation of the data that follow the given code symbol of the quantized latent representation of the data.

13. The method of claim 11 , wherein the one or more neural network layers are masked convolutional neural network layers.

14. The method of claim 10 , wherein processing the context outputs and the quantized latent representation of the entropy model by one or more neural network layers to generate the probability distribution parameters defining the entropy model comprises:

generating respective probability distribution parameters characterizing a respective code symbol probability distribution for each code symbol of the quantized latent representation of the data, comprising, for each code symbol of the quantized latent representation of the data:

processing an input comprising: (i) a context output for the code symbol of the quantized latent representation of the data, and (ii) the latent representation of the entropy model, using the one or more neural network layers to generate the probability distribution parameters characterizing the code symbol probability distribution for the code symbol of the quantized latent representation of the data.

15. The method of claim 1 , wherein parameters of neural network layers used to: (i) generate the compressed representation of the data, and (ii) generate a reconstruction of the data from the compressed representation of the data, are jointly trained using a machine learning training technique to optimize a rate-distortion performance measure.

16. The method of claim 10 , wherein parameters of: (i) the hyper-encoder neural network, and (ii) the neural network layers used to autoregressively process the code symbols of the quantized latent representation of the data to generate the plurality of context outputs, are jointly trained using a machine learning training technique to optimize a rate-distortion performance measure.

17. The method of claim 15 , wherein the rate-distortion performance measure comprises: (i) a first rate term based on a size of the entropy encoded representation of the latent representation of the data, (ii) a second rate term based on a size of the entropy encoded representation of the latent representation of the entropy model, and (iii) a distortion term based on a difference between the data and the reconstruction of the data.

18. A method implemented by a data processing apparatus, the method comprising:

obtaining entropy encoded representations of: (i) a latent representation of a set of data, and (ii) a latent representation of an entropy model used to entropy encode the latent representation of the data,

wherein the latent representation of the entropy model was determined by processing the latent representation of the data using a hyper-encoder neural network, and

wherein the entropy model is defined by one or more probability distribution parameters characterizing one or more code symbol probability distributions;

entropy decoding the latent representation of the data, comprising:

determining the probability distribution parameters defining the entropy model using the latent representation of the entropy model; and

entropy decoding the latent representation of the data using the entropy model; and

determining a reconstruction of the data from the latent representation of the data, comprising processing the latent representation of the data using a using a decoder neural network.

19. The method of claim 18 , wherein entropy decoding the latent representation of the data using the entropy model comprises, for one or more code symbols of a quantization of the latent representation of the data:

processing: (i) one or more preceding code symbols of the quantized latent representation of the data that precede the code symbol in an ordering of the code symbols of the quantized latent representation of the data, and (ii) a quantization of the latent representation of the entropy model, to generate a code symbol probability distribution corresponding to the code symbol of the quantized latent representation of the data; and

using the code symbol probability distribution corresponding to the code symbol of the quantized latent representation of the data to entropy decode the code symbol of the quantized latent representation of the data.

20. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

processing data using an encoder neural network to generate a latent representation of the data;

processing the latent representation of the data using a hyper-encoder neural network to generate a latent representation of an entropy model, wherein the entropy model is defined by one or more probability distribution parameters characterizing one or more code symbol probability distributions;

generating an entropy encoded representation of the latent representation of the entropy model;

generating an entropy encoded representation of the latent representation of the data using the latent representation of the entropy model, comprising:

determining the probability distribution parameters defining the entropy model using the latent representation of the entropy model; and

generating the entropy encoded representation of the latent representation of the data using the entropy model; and

determining a compressed representation of the data from the entropy encoded representations of: (i) the latent representation of the data and (ii) the latent representation of the entropy model used to entropy encode the latent representation of the data.

21. A system comprising:

one or more computers; and

one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

processing data using an encoder neural network to generate a latent representation of the data;

processing the latent representation of the data using a hyper-encoder neural network to generate a latent representation of an entropy model, wherein the entropy model is defined by one or more probability distribution parameters characterizing one or more code symbol probability distributions;

generating an entropy encoded representation of the latent representation of the entropy model;

generating an entropy encoded representation of the latent representation of the data using the latent representation of the entropy model, comprising:

determining the probability distribution parameters defining the entropy model using the latent representation of the entropy model; and

generating the entropy encoded representation of the latent representation of the data using the entropy model; and

determining a compressed representation of the data from the entropy encoded representations of: (i) the latent representation of the data and (ii) the latent representation of the entropy model used to entropy encode the latent representation of the data.