IP Library Granted Patent US 11,582,481
Granted Patent B2
US 11,582,481 · App. 17/039,643 · Granted Feb 14, 2023

Encoding and decoding image data

Inventors: Djordje Djokovic (London, GB); Ioannis Andreopoulos (London, GB); Ilya Fadeev (London, GB); Srdjan Grce (London, GB)
Assignee: ISIZE LIMITED
H04N19/59G06N3/04G06N3/08H04N19/80H04N19/85
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,582,481
App. No.
17/039,643
Granted
Feb 14, 2023
Kind
B2
Abstract

Certain aspects of the present disclosure provide techniques for encoding image data for one or more images. In one embodiment, a method includes the steps of downscaling the one or more images, and encoding the one or more downscaled images using an image codec. Another embodiment concerns a computer-implemented method of decoding encoded image data, and a computer-implemented method of encoding and decoding image data.

Claims (54)

1. A computer-implemented method of encoding image data for one or more images using an image codec, wherein the image codec comprises a downscaling process, the method comprising the steps of:

downscaling the one or more images in accordance with the initial downscaling process, using an artificial neural network comprising a plurality of layers of neurons;

subsequently encoding the one or more downscaled images using the image codec; and

iterating, until a stopping condition is reached, the steps of:

determining an importance value for a set of neurons or layers of neurons;

removing the neurons and/or layers with importance value less than a determined amount from the neural network; and

tuning the neural network in accordance with a cost function, wherein the cost function is arranged to determine a redetermined balance between the accuracy of the downscaling performed by neural network and the complexity of the neural network;

applying a monotonic scaling function to the weighting of the neurons of the neural network; and

tuning the neural network.

2. The method of claim 1 , wherein the one or more images are downscaled using one or more filters.

3. The method of claim 2 , wherein a filter of the one or more filters is an edge-detection filter.

4. The method of claim 2 , wherein a filter of the one or more filters is a blur filter.

5. The method of claim 2 , wherein the parameters of a filter used to downscale an image are determined using the results of the encoding of previous images by the image codec.

6. The method of claim 1 , wherein in the removal step, the layer with lowest importance value is removed.

7. The method of claim 1 , wherein the stopping condition is that the iterated steps have been performed a predetermined number of times.

8. The method of claim 1 , wherein the iterated steps and the applying and tuning steps are iterated until a further stopping condition is reached.

9. The method of claim 1 , wherein the step of downscaling the one or more images comprises the step of determining the processes to use to downscale an image using the results of the encoding of previous images by the image codec.

10. The method of claim 1 , wherein the encoded image data comprises data indicative of the downscaling performed.

11. The method of claim 1 , wherein the image codec is lossy.

12. A computer-implemented method of decoding encoded image data for one or more images, wherein the encoded image data was encoded using a method of encoding image data for one or more images using an image codec, wherein the image codec comprises a downscaling process, the method of encoding image data comprising the steps of:

downscaling the one or more images in accordance with the initial downscaling process using an artificial neural network comprising a plurality of layers of neurons; and

subsequently encoding the one or more downscaled images using the image codec;

iterating, until a stopping condition is reached, the steps of:

determining an importance value for a set of neurons or layers of neurons;

removing the neurons and/or layers with importance value less than a determined amount from the neural network; and

tuning the neural network in accordance with a cost function, wherein the cost function is arranged to determine a predetermined balance between the accuracy of the downscaling performed by neural network and the complexity of the neural network;

applying a monotonic scaling function to the weighting of the neurons of the neural network; and

tuning the neural network,

wherein the method of decoding encoded image data comprising the steps of:

decoding the encoded image data using the image codec to generate one or more downscaled images; and

upscaling the one or more images.

13. The method of claim 12 , wherein the encoded image data comprises data indicative of the downscaling performed, and the data indicative of the downscaling performed is used when upscaling the one or more images.

14. The method of claim 1 , wherein the image data is video data, the one or more images are frames of video, and the image codec is a video codec.

15. A computing device comprising:

a processor; and

a memory,

wherein the computing device is arranged to perform using the processor a method of encoding image data for one or more images using an image codec, wherein the image codec comprises a downscaling process, the method comprising the steps of:

downscaling the one or more images in accordance with the initial downscaling process using an artificial neural network comprising a plurality of layers of neurons;

subsequently encoding the one or more downscaled images using the image codec; and

iterating, until a stopping condition is reached, the steps of:

determining an importance value for a set of neurons or layers of neurons;

removing the neurons and/or layers with importance value less than a determined amount from the neural network; and

tuning the neural network in accordance with a cost function, wherein the cost function is arranged to determine a predetermined balance between the accuracy of the downscaling performed by neural network and the complexity of the neural network;

applying a monotonic scaling function to the weighting of the neurons of the neural network; and

tuning the neural network.

16. A non-transitory computer-readable medium comprising instructions that, when executed by a processor of a computing device, cause the computing device to perform a method of encoding image data for one or more images using an image codec, wherein the image codec comprises a downscaling process, the method comprising the steps of:

downscaling the one or more images in accordance with the initial downscaling process using an artificial neural network comprising a plurality of layers of neurons;

subsequently encoding the one or more downscaled images using the image codec; and

iterating, until a stopping condition is reached, the steps of:

determining an importance value for a set of neurons or layers of neurons;

removing the neurons and/or layers with importance value less than a determined amount from the neural network; and

tuning the neural network in accordance with a cost function, wherein the cost function is arranged to determine a predetermined balance between the accuracy of the downscaling performed by neural network and the complexity of the neural network;

applying a monotonic scaling function to the weighting of the neurons of the neural network; and

tuning the neural network.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE APPLICATION NUMBER TO 11445222 PREVIOUSLY RECORDED AT REEL: 67695 FRAME: 636. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 13, 2024
From: ISIZE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
Reel/Frame 067724/0694 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 11, 2024
From: ISIZE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
Reel/Frame 067695/0636 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2021
From: DJOKOVIC, DJORDJE; ANDREOPOULOS, IOANNIS; FADEEV, ILYA; GRCE, SRDJAN
To: ISIZE LIMITED
Reel/Frame 055689/0838 →
Continuity (4)
Continuation PCTGB2019051315 · May 14, 2019
Provisional Application 62682980 · Jun 10, 2018
Provisional Application 62672286 · May 16, 2018
Related Publication 20210021866A1 · Jan 21, 2021