METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING

1 - 96 .

97 . A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of:

receiving an input image at a first computer system;

encoding the input image using a first neural network to produce a latent representation; entropy encoding the latent representation;

transmitting the entropy encoded latent representation to a second computer system; entropy decoding the entropy encoded latent representation; and

decoding the latent representation using a second neural network to produce an output image, wherein the output image is an approximation of the input image;

determining a quantity based on a difference between the output image and the input image;

updating the parameters of the first neural network and the second neural network based on the determined quantity; and

repeating the above steps using a first set of input images to produce a first trained neural network and a second trained neural network;

wherein the entropy decoding of the entropy encoded latent representation is performed pixel by pixel; and

the order of the pixel by pixel decoding is additionally updated based on the determined quantity.

98 . The method of claim 97 , wherein the order of the pixel by pixel decoding is based on the latent representation.

99 . The method of claim 97 , wherein the entropy decoding of the entropy encoded latent comprises an operation based on previously decoded pixels.

100 . The method of claim 97 , wherein the determining of the order of the pixel by pixel decoding comprises ordering a plurality of the pixels of the latent representation in a directed acyclic graph.

101 . The method of claim 97 , wherein the determining of the order of the pixel by pixel decoding comprises operating on the latent representation with a plurality of adjacency matrices.

102 . The method of claim 97 , wherein the determining of the order of the pixel by pixel decoding comprises dividing the latent representation into a plurality of sub-images.

103 . The method of claim 102 , wherein the plurality of sub-images are obtained by convolving the latent representation with a plurality of binary mask kernels.

104 . The method of claim 97 , wherein the determining of the order of the pixel by pixel decoding comprises ranking a plurality of pixels of the latent representation based on the magnitude of a quantity associated with each pixel.

105 . The method of claim 104 , wherein the quantity associated with each pixel is the location or scale parameter associated with that pixel.

106 . The method of claim 104 , wherein the quantity associated with each pixel is additionally updated based on the evaluated difference.

107 . The method of claim 97 , wherein the determining of the order of the pixel by pixel decoding comprises a wavelet decomposition of a plurality of pixels of the latent representation.

108 . The method of claim 107 , wherein the order of the pixel by pixel decoding is based on the frequency components of the wavelet decomposition associated with the plurality of pixels.

109 . The method of claim 97 , further comprising the steps of:

encoding the latent representation using a fourth trained neural network to produce a hyper-latent representation;

transmitting the hyper-latent to the second computer system; and

decoding the hyper-latent using a fifth trained neural network, wherein the order of the pixel by pixel decoding is based on the output of the fifth trained neural network.

110 . A method for lossy image and video encoding, transmission and decoding, the method comprising the steps of:

receiving an input image at a first computer system;

encoding the input image using a first trained neural network to produce a latent representation;

entropy encoding the latent representation;

transmitting the entropy encoded latent representation to a second computer system; entropy decoding the entropy encoded latent representation; and

decoding the latent representation using a second trained neural network to produce an output image, wherein the output image is an approximation of the input image;

wherein the first trained neural network and the second trained neural network have been trained according to the method of claim 97 .

111 . A method for lossy image or video encoding and transmission, the method comprising the steps of:

receiving an input image at a first computer system;

encoding the input image using a first trained neural network to produce a latent representation;

entropy encoding the latent representation; and

transmitting the entropy encoded latent representation;

wherein the first trained neural network has been trained according to the method of claim 97 .

112 . (canceled)

113 . A data processing system configured to perform the method of claim 97 .

114 - 177 .