Granted Patent B2

US 12,347,149 · App. 17/950,569 · Granted Jul 1, 2025

Inventors: Ding Ding (Palo Alto, CA); Wei Wang (Palo Alto, CA); Shan Liu (Palo Alto, CA)

Assignee: TENCENT AMERICA LLC

G06T9/002G06N3/045G06T3/4046

Abstract

Claims (50)

1. A method of content-adaptive online training for end-to-end (E2E) neural image compression (NIC) using a neural network performed by at least one processor, the method comprising:

receiving, by an E2E NIC framework, an input image including one or more blocks;

preprocessing a first neural network of the E2E NIC framework, based on the one or more blocks;

computing updated parameters using the preprocessed first neural network, wherein the updated parameters include a learning rate and a number of steps, and wherein the learning rate and the number of steps are selected based on characteristics of the input image;

encoding the one or more blocks and the updated parameters;

updating the first neural network based on the encoded updated parameters; and

generating a compressed representation of the encoded one or more blocks using the updated first neural network.

2. The method according to claim 1 , further comprising:

splitting the input image into the one or more blocks; and

compressing the one or more blocks individually.

3. The method according to claim 1 , further comprising;

decoding the compressed representation using arithmetic decoding; and

generating a reconstructed image based on the decoded compressed representation using a second neural network.

4. The method according to claim 1 , further comprising compressing the updated parameters.

5. The method according to claim 1 , wherein the characteristics of the input image are one of a RGB variance of the input image and an RD performance of the input image.

6. The method according to claim 1 , wherein when preprocessing the first neural network, the first neural network is fine-tuned using the one or more blocks.

7. An apparatus for content-adaptive online training for end-to-end (E2E) neural image compression (NIC) using a neural network, the apparatus comprising:

at least one memory configured to store computer program code; and

at least one processor configured to read the computer program code and operate as instructed by the computer program code, the computer program code including:

receiving code configured to cause the at least one processor to receive, by an E2E NIC framework, an input including one or more blocks;

preprocessing code configured to cause the at least one processor to preprocess a first neural network of the E2E NIC framework, based on the one or more blocks;

computing code configured to cause the at least one processor to compute updated parameters using the preprocessed first neural network, wherein the updated parameters include a learning rate and a number of steps, and wherein the learning rate and the number of steps are selected based on characteristics of the input image;

encoding code configured to cause the at least one processor to encode the one or more blocks and the updated parameters;

updating code configured to cause the at least one processor to update the first neural network based on the encoded updated parameters; and

first generating code configured to cause the at least one processor to generate a compressed representation of the encoded one or more blocks using the updated first neural network.

8. The apparatus of claim 7 , the computer program code further including:

splitting code configured to cause the at least one processor to split the input image into the one or more blocks; and

compressing code configured to cause the at least one processor to compress the one or more blocks individually.

9. The apparatus of claim 7 , the computer program code further including:

decoding code configured to cause the at least one processor to decode the compressed representation using arithmetic decoding; and

second generating code configured to cause the at least one processor to generate a reconstructed image based on the decoded compressed representation using a second neural network.

10. The apparatus of claim 7 , the computer program code further including compressing code configured to cause the at least one processor to compress the updated parameters.

11. The apparatus of claim 7 , wherein the characteristics of the input image are one of a RGB variance of the input image and an RD performance of the input image.

12. The apparatus of claim 7 , wherein when preprocessing the first neural network, the first neural network is fine-tuned using the one or more blocks.

13. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor of an apparatus for content-adaptive online training for end-to-end (E2E) neural image compression (NIC) using a neural network, cause the at least one processor to:

receive, by an E2E NIC framework, an input image including one or more blocks;

preprocess a first neural network of the E2E NIC framework, based on the one or more blocks;

compute updated parameters using the preprocessed first neural network;

encode the one or more blocks and the updated parameters, wherein the updated parameters include a learning rate and a number of steps, and wherein the learning rate and the number of steps are selected based on characteristics of the input image;

update the first neural network based on the encoded updated parameters; and

generate a compressed representation of the encoded one or more blocks using the updated first neural network.

14. The non-transitory computer-readable medium of claim 13 , wherein the instructions further cause the at least one processor to:

split the input image into the one or more blocks; and

compress the one or more blocks individually.

15. The non-transitory computer-readable medium of claim 13 , wherein the instructions further cause the at least one processor to:

decode the compressed representation using arithmetic decoding; and

generate a reconstructed image based on the decoded compressed representation using a second neural network.

16. The non-transitory computer-readable medium of claim 13 , wherein the instructions further cause the at least one processor to compress the updated parameters.

17. The non-transitory computer-readable medium of claim 13 , wherein the characteristics of the input image are one of a RGB variance of the input image and an RD performance of the input image.

18. The non-transitory computer-readable medium of claim 13 , wherein when preprocessing the first neural network, the first neural network is fine-tuned using the one or more blocks.

Assignments (1)

Continuity (2)

References Cited (19)