Neural image compression with controllable spatial bit allocation

1. A processor-implemented method, comprising:

receiving, at an encoder of an artificial neural network, an image and a spatial segmentation map corresponding to the image, the spatial segmentation map indicating one or more regions of interest;

learning, at an auxiliary neural network, a quantization bin size based on the spatial segmentation map;

generating a learned quantization bin size in accordance with the learning; and

compressing, via the encoder, the image according to a controllable spatial bit allocation, the controllable spatial bit allocation is based on the learned quantization bin size.

2. The processor-implemented method of claim 1 , further comprising:

quantizing a latent representation of the image and the spatial segmentation map based on the learned quantization bin size; and

transmitting the quantized latent representation of the image and the quantized segmentation map to a decoder.

3. The processor-implemented method of claim 1 , in which the learned quantization bin size provides a different bit allocation for different regions of interest.

4. The processor-implemented method of claim 1 , in which a bit allocation for non-regions of interest is controlled via a reduction factor.

5. The processor-implemented method of claim 1 , in which the learned quantization bin size is adjusted based on a scaling parameter.

6. The processor-implemented method of claim 1 , in which the spatial segmentation map comprises a binary segmentation map.

7. A processor-implemented method, comprising:

receiving, at a decoder of an artificial neural network, a bit stream from an encoder;

decoding, via the decoder, the bit stream to recover a quantized latent representation of an image corresponding to a spatial segmentation map, the spatial segmentation map indicating one or more regions of interest;

applying reciprocal gain control elements to the quantized latent representation to produce the spatial segmentation map, the reciprocal gain control elements having a gain value corresponding to a learned quantization bin size that was learned at an auxiliary neural network; and

generating, via the decoder, a reconstructed image based on the spatial segmentation map.

8. The processor-implemented method of claim 7 , in which the learned quantization bin size is based on the one or more regions of interest to control a rate-distortion tradeoff for each spatial location of the image.

9. The processor-implemented method of claim 7 , in which the reciprocal gain control elements are configured based on the spatial segmentation map.

10. The processor-implemented method of claim 7 , in which the learned quantization bin size provides a different bit allocation for different regions of interest.

11. An apparatus, comprising:

memory; and

at least one processor coupled to the memory, the at least one processor configured:

to receive, at an encoder of an artificial neural network, an image and a spatial segmentation map corresponding to the image, the spatial segmentation map indicating one or more regions of interest;

to learn, at an auxiliary neural network, a quantization bin size based on the spatial segmentation map;

to generate a learned quantization bin size in accordance with the learning; and

to compress, via the encoder, the image according to a controllable spatial bit allocation, the controllable spatial bit allocation is based on a learned quantization bin size.

12. The apparatus of claim 11 , in which the at least one processor is further configured:

to quantize a latent representation of the image and the spatial segmentation map based on the learned quantization bin size; and

to transmit the quantized latent representation of the image and the quantized segmentation map to a decoder.

13. The apparatus of claim 11 , in which the learned quantization bin size provides a different bit allocation for different regions of interest.

14. The apparatus of claim 11 , in which the at least one processor is further configured to control a bit allocation for non-regions of interest via a reduction factor.

15. The apparatus of claim 11 , in which the at least one processor is further configured to adjust the learned quantization bin size based on a scaling parameter.

16. The apparatus of claim 11 , in which the spatial segmentation map comprises a binary segmentation map.

17. An apparatus, comprising:

memory; and

at least one processor coupled to the memory, the at least one processor configured:

to receive, at a decoder of an artificial neural network, a bit stream from an encoder;

to decode, via the decoder, the bit stream to recover a quantized latent representation of an image corresponding to a spatial segmentation map, the spatial segmentation map indicating one or more regions of interest;

to apply reciprocal gain control elements to the quantized latent representation to produce the spatial segmentation map, the reciprocal gain control elements having a gain value corresponding to a learned quantization bin size that was learned at an auxiliary neural network; and

to generate, via the decoder, a reconstructed image based on the spatial segmentation map.

18. The apparatus of claim 17 , in which the learned quantization bin size is based on the one or more regions of interest to control a rate-distortion tradeoff for each spatial location of the image.

19. The apparatus of claim 17 , in which the at least one processor is further configured to configure the reciprocal gain control elements based on the spatial segmentation map.

20. The apparatus of claim 17 , in which the learned quantization bin size provides a different bit allocation for different regions of interest.