IP Library › Granted Patent US 12,142,014
Granted Patent B2
US 12,142,014 · App. 17/430,987 · Granted Nov 12, 2024

Method, an apparatus and a computer program product for video encoding and video decoding

Inventors: Caglar Aytekin (Tampere, FI); Francesco Cricri (Tampere, FI); Mikko Honkala (Espoo, FI)
Assignee: Nokia Technologies Oy
G06T9/002G06N3/08H04N19/132H04N19/15H04N19/196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,142,014
App. No.
17/430,987
Granted
Nov 12, 2024
Kind
B2
Abstract

The embodiments relate to a method comprising compressing input data (I) by means of at least a neural network (E, 310 ); determining a compression rate for data compression; miming the neural network (E, 310 ) with the input data (I) to produce an output data (c); removing a number of elements from the output data (c) according to the compression rate to result in a reduced form of the output data (me); and providing the reduced form of the output data (me) and the compression rate to a decoder (D, 320 ). The embodiments also relate to a method comprising receiving input data (me) for decompression; decompressing the input data (me) by means of at least a neural network (D, 320 ); determining a decompression rate for decompressing the input data (me); miming the neural network (D, 320 ) with input data (me) to produce a decompressed output data (ĩ); padding a number of elements to the compressed input data (me) according to the decompression rate to produce an output data (ĩ); and providing the output data (ĩ).

Claims (51)

1. A method, comprising:

compressing input data by means of at least a neural network;

determining a compression rate for data compression;

running the neural network with the input data to produce an output data;

removing a number of elements from the output data according to the compression rate to generate a reduced form of the output data;

wherein the compression rate is associated with a mask, based on which the number of elements are removed; and

providing the reduced form of the output data and the compression rate to a decoder.

2. The method according to claim 1 , wherein the neural network is trained with multiple masks associated with different compression rates.

3. The method according to claim 2 , wherein each mask of the multiple masks include a pattern of N ones, and wherein N is different for each compression rate.

4. The method according to claim 3 , wherein first N elements of a dimension of the reduced form of the output data are equal to one and the rest of that dimension are equal to zero.

5. The method according to claim 2 , further comprising:

selecting N randomly from a uniform distribution during training.

6. The method according to claim 2 , further comprising:

selecting N randomly from a non-uniform distribution emphasizing different compression rates or ranges of compression rates.

7. The method according to claim 2 , further comprising:

training the neural network with different values of N, and averaging losses associated with the different values of N for a weight update.

8. A method comprising:

receiving input data for decompression;

decompressing the input data by means of at least a neural network;

determining a decompression rate for decompressing the input data;

running the neural network with the input data to produce a decompressed output data;

padding a number of elements to the decompressed output data according to the decompression rate to produce an output data;

wherein the decompression rate is associated with a mask, based on which the number of elements are padded; and

providing the output data.

9. An apparatus comprising:

at least one processor; and

at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to:

compress input data, wherein compression comprises at least a neural network;

determine a compression rate for data compression;

run the neural network with the input data to produce an output data;

remove a number of elements from the output data according to the compression rate to generate a reduced form of the output data;

wherein the compression rate is associated with a mask, based on which the number of elements are removed; and

provide the reduced form of the output data and the compression rate to a decoder.

10. The apparatus according to claim 9 , wherein the neural network is trained with multiple masks associated with different compression rates.

11. The apparatus according to claim 10 , wherein each mask of the multiple masks include a pattern of N ones, and wherein N is different for each compression rate.

12. The apparatus according to claim 11 , wherein first N elements of a dimension of the reduced form of the output data are equal to one and the rest of that dimension are equal to zero.

13. The apparatus according to claim 10 , wherein the at least one memory and the instructions are further configured to, with the at least one processor, further cause the apparatus to: select N randomly from a uniform distribution during training.

14. The apparatus according to claim 10 , wherein the at least one memory and the instructions are further configured to, with the at least one processor, further cause the apparatus to: select N randomly from a non-uniform distribution emphasizing different compression rates or ranges of compression rates.

15. The apparatus according to claim 10 , wherein the at least one memory and the instructions are further configured to, with the at least one processor, further cause the apparatus to: train the neural network with different values of N, and average losses associated with the different values of N for a weight update.

16. The apparatus according to claim 10 , wherein the at least one memory and the instructions are further configured to, with the at least one processor, further cause the apparatus to: train the neural network with different values of N, and weight the losses based on an inverse frequency of elements a code that are not removed during training.

17. The apparatus according to claim 10 , wherein the at least one memory and the instructions are further configured to, with the at least one processor, further cause the apparatus to: train the neural network with different values of N and have at least one fixed weight based on a value of N.

18. An apparatus comprising:

at least one processor; and

at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to:

receive input data for decompression;

decompress input data, wherein decompression comprises of at least a neural network;

determine a decompression rate for data decompression;

run the neural network with the input data to produce a decompressed output data;

pad a number of elements to the decompressed output data according to the decompression rate to produce an output data;

wherein the decompression rate is associated with a mask, based on which the number of elements are padded; and

provide the output data.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2021
From: AYTEKIN, CAGLAR; CRICRI, FRANCESCO; HONKALA, MIKKO
To: NOKIA TECHNOLOGIES OY
Reel/Frame 058044/0897 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2021
From: AYTEKIN, CAGLAR; CRICRI, FRANCESCO; HONKALA, MIKKO
To: NOKIA TECHNOLOGIES OY
Reel/Frame 057708/0687 →
Priority Claims (1)
FI 20195114 · Feb 15, 2019 · national
Continuity (1)
Related Publication 20220164995A1 · May 26, 2022