Granted Patent B2

US 12,323,593 · App. 18/230,361 · Granted Jun 3, 2025

Inventors: Chri Besenbruch (London, GB); Ciro Cursio (London, GB); Christopher Finlay (London, GB); Vira Koshkina (London, GB); Alexander Lytchier (London, GB); Jan Xu (London, GB); Arsalan Zafar (London, GB)

Assignee: DEEP RENDER LTD.

H04N19/126G06N3/045G06N3/084G06T3/4046G06T9/002G06V10/774H04N19/13

Abstract

Claims (20)

1. A computer implemented method of training a first neural network and a second neural network based on training images in which each respective training image includes human scored data relating to a perceived level of distortion in the respective training image as evaluated by a group of humans, the first and second neural networks being for use in lossy image or video compression, transmission and decoding, the method including the steps of:

(i) generating a set of training images by forward passing a set of images through a trained AI-based compression pipeline, each training image having one or more artefacts representative of artefacts introduced by the trained AI-based compression pipeline, and by human scoring each generated training image based on a perceived level of distortion;

(ii) receiving an input training image from the generated set of training images;

(iii) encoding the input training image using the first neural network, to produce a latent representation;

(iv) quantizing the latent representation to produce a quantized latent;

(v) using the second neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image;

(vi) evaluating a loss function based on differences between the output image and the input training image;

(vii) evaluating a gradient of the loss function;

(viii) back-propagating the gradient of the loss function through the second neural network and through the first neural network, to update weights of the second neural network and of the first neural network;

(ix) repeating steps (i) to (viii) using a set of training images, to produce a trained first neural network and a trained second neural network; and

(x) storing the weights of the trained first neural network and of the trained second neural network;

wherein the loss function includes a weighted sum of a rate term and a distortion term, and wherein the distortion term is a function based on human scored data of the respective training image from the generated set of training images.

2. The method of claim 1 , wherein the loss function is evaluated as a weighted sum of differences between the output image and the input training image, and the estimated bits of the quantized image latents.

3. The method of claim 1 , wherein at least one thousand training images are used.

4. The method of claim 1 , wherein the training images include a plurality of distortion types.

5. The method of claim 1 , wherein the training images include at least one distortion type corresponding to one or more distortion types introduced using AI-based compression encoder-decoder pipelines.

6. The method of claim 1 , wherein the human scored data is based on human labelled data.

7. The method of claim 1 , wherein in step (v) the loss function includes a component that represents the human visual system.

8. The method of claim 7 , wherein a distortion term of the loss function comprises a MSE and/or PSNR term.

9. The method of claim 1 , wherein said generating comprises generating the training set of images by forward passing the set of images through the trained AI-based compression pipeline at different time steps in the training of the trained AI-based compression pipeline.

Assignments (1)

Priority Claims (12)

Continuity (6)

References Cited (64)

Cited By (3)

US 12,380,624 US 12,387,466 US 12,647,611