Alias-free compression of content using artificial neural networks
Certain aspects of the present disclosure provide techniques and apparatus for encoding content using a neural network. An example method generally includes encoding video content into a latent space representation through an encoder implemented by a first machine learning model. A code is generated by upsampling the latent space representation of the video content. A prior is calculated based on a conditional probability of obtaining the upsampled latent space representation conditioned by the latent space representation of the video content. A compressed version of the video content is generated based on a probabilistic model implemented by a second machine learning model, the generated code, and the calculated prior, and the compressed version of the video content is output for transmission.
1. A computer-implemented method, comprising:
encoding video content into a latent space representation through an encoder implemented by a first machine learning model;
generating a code by upsampling the latent space representation of the video content;
calculating a prior based on a conditional probability of obtaining the upsampled latent space representation conditioned by the latent space representation of the video content;
generating a compressed version of the video content based on a probabilistic model implemented by a second machine learning model, the generated code, and the calculated prior; and
outputting the compressed version of the video content.
2. The method of claim 1 , wherein upsampling the latent space representation of the video content comprises upsampling the latent space representation of the video content based on patches in the video content.
3. The method of claim 2 , wherein a size of each patch is based on a defined factor by which the video content is upsampled.
4. The method of claim 3 , wherein the size of each patch comprises dimensions of the defined factor in pixels on a horizontal axis in the video content by the defined factor in pixels on a vertical axis in the video content.
5. The method of claim 2 , wherein generating the code comprises upsampling each respective patch into a code in an upsampled latent space based on a correlation between pixels in the respective patch.
6. The method of claim 1 , wherein the first machine learning model comprises a first artificial neural network and wherein the second machine learning model comprises a second artificial neural network.
7. A computer-implemented method, comprising:
receiving a compressed version of an encoded video content, the encoded video content being based on an upsampled latent space representation of source video content;
recovering a latent space representation of the source video content through a probabilistic model implemented by a first machine learning model based on a prior defined as a conditional probability of obtaining the upsampled latent space representation conditioned by the latent space representation of the source video content;
downsampling the recovered latent space representation of the source video content;
reconstructing the encoded video content from the downsampled recovered latent space representation of the source video content through a decoder implemented by a second machine learning model; and
outputting the reconstruction of the encoded video content.
8. The method of claim 7 , wherein the compressed version of the encoded video content comprises a bitstream downsampled from an encoded version of the upsampled latent space representation of the source video content, and wherein the upsampled latent space representation comprises a version of the latent space representation of the source video content upsampled based on patches in the source video content.
9. The method of claim 8 , wherein a size of each patch is based on a defined factor by which the source video content is upsampled.
10. The method of claim 9 , wherein the size of each patch comprises dimensions of the defined factor in pixels on a horizontal axis in the source video content by the defined factor in pixels on a vertical axis in the source video content.
11. The method of claim 7 , wherein the first machine learning model comprises a first artificial neural network and wherein the second machine learning model comprises a second artificial neural network.
12. An apparatus, comprising:
a memory having executable instructions stored thereon; and
a processor configured to execute the executable instructions in order to cause the apparatus to:
encode video content into a latent space representation through an encoder implemented by a first machine learning model;
generate a code by upsampling the latent space representation of the video content;
calculate a prior based on a conditional probability of obtaining the upsampled latent space representation conditioned by the latent space representation of the video content;
generate a compressed version of the video content based on a probabilistic model implemented by a second machine learning model, the generated code, and the calculated prior; and
output the compressed version of the video content.
13. The apparatus of claim 12 , wherein in order to upsample the latent space representation of the video content, the processor is configured to cause the apparatus to upsample the latent space representation of the video content based on patches in the video content.
14. The apparatus of claim 13 , wherein a size of each patch is based on a defined factor by which the video content is upsampled.
15. The apparatus of claim 14 , wherein the size of each patch comprises dimensions of the defined factor in pixels on a horizontal axis in the video content by the defined factor in pixels on a vertical axis in the video content.
16. The apparatus of claim 13 , wherein in order to generate the code, the processor is configured to cause the apparatus to upsample each respective patch into a code in an upsampled latent space based on a correlation between pixels in the respective patch.
17. The apparatus of claim 12 , wherein the first machine learning model comprises a first artificial neural network and wherein the second machine learning model comprises a second artificial neural network.
18. An apparatus, comprising:
a memory having executable instructions stored thereon; and
a processor configured to execute the executable instructions in order to cause the apparatus to:
receive a compressed version of an encoded video content, the encoded video content being based on an upsampled latent space representation of source video content;
recover a latent space representation of the source video content through a probabilistic model implemented by a first machine learning model based on a prior defined as a conditional probability of obtaining the upsampled latent space representation conditioned by the latent space representation of the source video content;
downsample the recovered latent space representation of the source video content;
reconstruct the encoded video content from the downsampled recovered latent space representation of the source video content through a decoder implemented by a second machine learning model; and
output the reconstruction of the encoded video content.
19. The apparatus of claim 18 , wherein the compressed version of the encoded video content comprises a bitstream downsampled from an encoded version of the upsampled latent space representation of the source video content, and wherein the upsampled latent space representation comprises a version of the latent space representation of the source video content upsampled based on patches in the source video content.
20. The apparatus of claim 19 , wherein a size of each patch is based on a defined factor by which the source video content is upsampled.
21. The apparatus of claim 20 , wherein the size of each patch comprises the defined factor in pixels on a horizontal axis in the source video content by the defined factor in pixels on a vertical axis in the source video content.
22. The apparatus of claim 18 , wherein the first machine learning model comprises a first artificial neural network and wherein the second machine learning model comprises a second artificial neural network.