Method and apparatus for multi-rate neural image compression with stackable nested model structures

1. A method of multi-rate neural image compression with stackable nested model structures, the method being performed by at least one processor, and the method comprising:

iteratively stacking, on a first prior set of weights of a first neural network corresponding to a prior hyperparameter, a first plurality of sets of weights of a first plurality of stackable neural networks corresponding to a current hyperparameter, wherein the first prior set of weights of the first neural network remains unchanged;

encoding an input image to obtain an encoded representation, using the first prior set of weights of the first neural network on which the first plurality of sets of weights of the first plurality of stackable neural networks is stacked; and

encoding the obtained encoded representation to determine a compressed representation.

2. The method of claim 1 , further comprising:

iteratively stacking, on a second prior set of weights of a second neural network corresponding to the prior hyperparameter, a second plurality of sets of weights of a second plurality of stackable neural networks corresponding to the current hyperparameter, wherein the second prior set of weights of the second neural network remains unchanged;

decoding the determined compressed representation to determine a recovered representation; and

decoding the determined recovered representation to reconstruct an output image, using the second prior set of weights of the second neural network on which the second plurality of sets of weights of the second plurality of stackable neural networks is stacked.

3. The method of claim 2 , wherein the first neural network and the second neural network are trained by updating a first initial set of weights of the first neural network and a second initial set of weights of the second neural network, to optimize a rate-distortion loss that is determined based on the input image, the output image and the compressed representation.

4. The method of claim 3 , wherein the first neural network and the second neural network are further trained by iteratively stacking, on the first prior set of weights of the first neural network, the first plurality of sets of weights of the first plurality of stackable neural networks corresponding to the current hyperparameter, wherein the first prior set of weights of the first neural network remains unchanged.

5. The method of claim 4 , wherein the first neural network and the second neural network are further trained by iteratively stacking, on the second prior set of weights of the second neural network, the second plurality of sets of weights of the second plurality of stackable neural networks corresponding to the current hyperparameter, wherein the second prior set of weights of the second neural network remains unchanged.

6. The method of claim 5 , wherein the first neural network and the second neural network are further trained by updating the stacked first plurality of sets of weights of the first plurality of stackable neural networks, and the stacked second plurality of sets of weights of the second plurality of stackable neural networks, to optimize the rate-distortion loss.

7. The method of claim 2 , wherein one or more of the first plurality of sets of weights of the first plurality of stackable neural networks and the second plurality of sets of weights of the second plurality of stackable neural networks do not correspond to the current hyperparameter.

8. An apparatus for multi-rate neural image compression with stackable nested model structures, the apparatus comprising:

at least one memory configured to store program code; and

at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:

first stacking code configured to cause the at least one processor to iteratively stack, on a first prior set of weights of a first neural network corresponding to a prior hyperparameter, a first plurality of sets of weights of a first plurality of stackable neural networks corresponding to a current hyperparameter, wherein the first prior set of weights of the first neural network remains unchanged;

first encoding code configured to cause the at least one processor to encode an input image to obtain an encoded representation, using the first prior set of weights of the first neural network on which the first plurality of sets of weights of the first plurality of stackable neural networks is stacked; and

second encoding code configured to cause the at least one processor to encode the obtained encoded representation to determine a compressed representation.

9. The apparatus of claim 8 , wherein the program code further comprises:

second stacking code configured to cause the at least one processor to iteratively stack, on a second prior set of weights of a second neural network corresponding to the prior hyperparameter, a second plurality of sets of weights of a second plurality of stackable neural networks corresponding to the current hyperparameter, wherein the second prior set of weights of the second neural network remains unchanged;

first decoding code configured to cause the at least one processor to decode the determined compressed representation to determine a recovered representation; and

second decoding code configured to cause the at least one processor to decode the determined recovered representation to reconstruct an output image, using the second prior set of weights of the second neural network on which the second plurality of sets of weights of the second plurality of stackable neural networks is stacked.

10. The apparatus of claim 9 , wherein the first neural network and the second neural network are trained by updating a first initial set of weights of the first neural network and a second initial set of weights of the second neural network, to optimize a rate-distortion loss that is determined based on the input image, the output image and the compressed representation.

11. The apparatus of claim 10 , wherein the first neural network and the second neural network are further trained by iteratively stacking, on the first prior set of weights of the first neural network, the first plurality of sets of weights of the first plurality of stackable neural networks corresponding to the current hyperparameter, wherein the first prior set of weights of the first neural network remains unchanged.

12. The apparatus of claim 11 , wherein the first neural network and the second neural network are further trained by iteratively stacking, on the second prior set of weights of the second neural network, the second plurality of sets of weights of the second plurality of stackable neural networks corresponding to the current hyperparameter, wherein the second prior set of weights of the second neural network remains unchanged.

13. The apparatus of claim 12 , wherein the first neural network and the second neural network are further trained by updating the stacked first plurality of sets of weights of the first plurality of stackable neural networks, and the stacked second plurality of sets of weights of the second plurality of stackable neural networks, to optimize the rate-distortion loss.

14. The apparatus of claim 9 , wherein one or more of the first plurality of sets of weights of the first plurality of stackable neural networks and the second plurality of sets of weights of the second plurality of stackable neural networks do not correspond to the current hyperparameter.

15. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor for multi-rate neural image compression with stackable nested model structures, cause the at least one processor to:

iteratively stack, on a first prior set of weights of a first neural network corresponding to a prior hyperparameter, a first plurality of sets of weights of a first plurality of stackable neural networks corresponding to a current hyperparameter, wherein the first prior set of weights of the first neural network remains unchanged;

encode an input image to obtain an encoded representation, using the first prior set of weights of the first neural network on which the first plurality of sets of weights of the first plurality of stackable neural networks is stacked; and

encode the obtained encoded representation to determine a compressed representation.

16. The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

iteratively stack, on a second prior set of weights of a second neural network corresponding to the prior hyperparameter, a second plurality of sets of weights of a second plurality of stackable neural networks corresponding to the current hyperparameter, wherein the second prior set of weights of the second neural network remains unchanged;

decode the determined compressed representation to determine a recovered representation; and

decode the determined recovered representation to reconstruct an output image, using the second prior set of weights of the second neural network on which the second plurality of sets of weights of the second plurality of stackable neural networks is stacked.

17. The non-transitory computer-readable medium of claim 16 , wherein the first neural network and the second neural network are trained by updating a first initial set of weights of the first neural network and a second initial set of weights of the second neural network, to optimize a rate-distortion loss that is determined based on the input image, the output image and the compressed representation.

18. The non-transitory computer-readable medium of claim 17 , wherein the first neural network and the second neural network are further trained by iteratively stacking, on the first prior set of weights of the first neural network, the first plurality of sets of weights of the first plurality of stackable neural networks corresponding to the current hyperparameter, wherein the first prior set of weights of the first neural network remains unchanged.

19. The non-transitory computer-readable medium of claim 18 , wherein the first neural network and the second neural network are further trained by iteratively stacking, on the second prior set of weights of the second neural network, the second plurality of sets of weights of the second plurality of stackable neural networks corresponding to the current hyperparameter, wherein the second prior set of weights of the second neural network remains unchanged.

20. The non-transitory computer-readable medium of claim 19 , wherein the first neural network and the second neural network are further trained by updating the stacked first plurality of sets of weights of the first plurality of stackable neural networks, and the stacked second plurality of sets of weights of the second plurality of stackable neural networks, to optimize the rate-distortion loss.