Granted Patent B2

US 12,028,525 · App. 18/230,312 · Granted Jul 2, 2024

Inventors: Chri Besenbruch (London, GB); Ciro Cursio (London, GB); Christopher Finlay (London, GB); Vira Koshkina (London, GB); Alexander Lytchier (London, GB); Jan Xu (London, GB); Arsalan Zafar (London, GB)

Assignee: DEEP RENDER LTD.

H04N19/126G06N3/045G06N3/084G06T3/4046G06T9/002G06V10/774H04N19/13

Abstract

Claims (38)

1. A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of:

(i) receiving an input image at a first computer system;

(ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation;

(iii) quantizing the latent representation using the first computer system to produce a quantized latent;

(iv) entropy encoding the quantized latent into a bitstream, using the first computer system;

(v) transmitting the bitstream to a second computer system;

(vi) the second computer system entropy decoding the bitstream to produce the quantized latent;

(vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image;

wherein steps (ii) to (vii) are executed wholly or partially in a frequency domain; and

wherein the first neural network is configured to perform one or both of: a spectral convolution, and applying a spectral-specific activation function.

2. The method of claim 1 , wherein integral transforms to and from the frequency domain are used.

3. The method of claim 2 , wherein the integral transforms comprise one or more of Fourier Transforms, or Hartley Transforms, or Wavelet Transforms, or Chirplet Transforms, or Sine and Cosine Transforms, or Mellin Transforms, or Hankel Transforms, or Laplace Transforms.

4. The method of claim 1 , comprising, downsampling the input image by: dividing the input image into a plurality of blocks that are concatenated in a separate dimension; applying a convolution operation with a 1×1 kernel to reduce a number of channels by half; and upsampling by following a reverse and mirrored methodology.

5. The method of claim 1 , wherein for image decomposition, stacking is performed.

6. The method of claim 1 , wherein for image reconstruction, stitching is performed.

7. A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of:

(i) receiving an input image at a first computer system;

(ii) encoding the input image using a first trained neural network, using the first computer system, to produce a y latent representation;

(iii) quantizing the y latent representation using the first computer system to produce a quantized y latent;

(iv) encoding the quantized y latent using a third trained neural network, using the first computer system, to produce a z latent representation;

(v) quantizing the z latent representation using the first computer system to produce a quantized z latent;

(vi) entropy encoding the quantized z latent into a second bitstream, using the first computer system;

(vii) the first computer system processing the quantized z latent using a fourth trained neural network to obtain probability distribution parameters of each element of the quantized y latent, wherein the probability distribution of the quantized y latent is assumed to be represented by a probability distribution of each element of the quantized y latent;

(viii) entropy encoding the quantized y latent, using the obtained probability distribution parameters of each element of the quantized y latent, into a first bitstream, using the first computer system;

(ix) transmitting the first bitstream and the second bitstream to a second computer system;

(x) the second computer system entropy decoding the second bitstream to produce the quantized z latent;

(xi) the second computer system processing the quantized z latent using a trained neural network identical to the fourth trained neural network to obtain the probability distribution parameters of each element of the quantized y latent;

(xii) the second computer system using the obtained probability distribution parameters of each element of the quantized y latent, together with the first bitstream, to obtain the quantized y latent;

(xiii) the second computer system using a second trained neural network to produce an output image from the quantized y latent, wherein the output image is an approximation of the input image;

wherein steps (ii) to (xiii) are executed wholly or partially in a frequency domain; and

wherein the first neural network is configured to perform one or both of: a spectral convolution, and applying a spectral-specific activation function.

8. The method of claim 7 , wherein integral transforms to and from the frequency domain are used.

9. The method of claim 8 , wherein the integral transforms comprise one or more of: Fourier Transforms, or Hartley Transforms, or Wavelet Transforms, or Chirplet Transforms, or Sine and Cosine Transforms, or Mellin Transforms, or Hankel Transforms, or Laplace Transforms.

10. The method of claim 7 , wherein the first trained neural network is configured to perform a spectral convolution.

11. The method of claim 7 , wherein one or more activation functions of the first trained neural network comprise spectral specific activation functions.

12. The method of claim 7 , comprising downsampling the input image by: dividing the input image into a plurality of blocks that are concatenated in a separate dimension; applying a convolution operation with a 1×1 kernel to reduce the number of channels by half; and upsampling by following a reverse and mirrored methodology.

13. The method of claim 7 , wherein for image decomposition, stacking is performed.

14. The method of claim 7 , wherein for image reconstruction, stitching is performed.

Assignments (1)

Priority Claims (12)

Continuity (6)