Encoding and decoding images using differentiable JPEG compression

1 . A computer-implemented method for encoding and decoding images using differentiable Joint Photographic Experts Group (JPEG) compression, comprising:

converting an image from a Red-Green-Blue (RGB) color space to a Luminance-Chrominance (YCbCr) color space to obtain a luminance channel and chrominance channels, and applying chroma subsampling to the chrominance channels to reduce resolution;

dividing the YCbCr image into pixel blocks and performing a discrete cosine transform (DCT) on each of the pixel blocks to obtain DCT coefficients;

quantizing the DCT coefficients using a scaled quantization table (SQT) to generate quantized DCT coefficients and reduce precision by:

computing SQT values with a differentiable floor function that utilizes a polynomial rounding function to approximate a floor of numerical values for the SQT;

maintaining non-zero gradients of the SQT values by utilizing a differentiable clipping function that computes a soft approximation of the SQT values based on an integer range of the JPEG compression and a scale parameter; and

applying the SQT to DCT patches of the image by rounding a result from dividing DCT coefficients with a corresponding value in the SQT;

encoding the quantized DCT coefficients using lossless entropy coding to generate a compressed JPEG file and decoding the compressed JPEG file by reversing the lossless entropy coding to obtain the quantized DCT coefficients;

dequantizing the quantized DCT coefficients using the scaled quantization table to generate dequantized DCT coefficients to restore the precision;

applying an inverse discrete cosine transform (IDCT) to the dequantized DCT coefficients to convert the dequantized DCT coefficients back to a spatial domain; and

upsampling the chrominance channels to an original resolution, and converting the YCbCr image back to the RGB color space.

2 . The method of claim 1 , wherein converting the RGB image to the YCbCr color space comprises applying a pixel-wise affine transformation.

3 . The method of claim 1 , wherein the chroma subsampling involves anti-aliasing using a 2D convolution followed by downsampling.

4 . The method of claim 1 , wherein a scale factor for the quantization table is computed based on a JPEG quality parameter, and the quantization table is scaled by the scale factor.

5 . The method of claim 1 , wherein the encoding the quantized DCT coefficients using lossless entropy coding comprises arranging the coefficients in a zigzag order, performing run-length encoding, and applying Huffman coding.

6 . The method of claim 1 , wherein the decoding further comprises clipping output pixel values to ensure they remain within a valid range by utilizing a product of the output pixel values and the scale parameter when the pixel values are outside the valid range.

7 . The method of claim 1 , further comprising using a straight-through estimator (STE) technique during the encoding and the decoding to execute accurate gradient propagation for machine learning applications.

8 . A system for encoding and decoding images using a differentiable JPEG approach, comprising:

a processor device; and

a memory storing instructions that, when executed by the processor device, cause the system to:

convert an image from a Red-Green-Blue (RGB) color space to a Luminance-Chrominance (YCbCr) color space to obtain a luminance channel and chrominance channels, and apply chroma subsampling to the chrominance channels to reduce resolution;

divide the YCbCr image into pixel blocks and performing a discrete cosine transform (DCT) on each of the pixel blocks to obtain DCT coefficients;

quantize the DCT coefficients using a scaled quantization table (SQT) to generate quantized DCT coefficients and reduce precision by:

computing SQT values with a differentiable floor function that utilizes a polynomial rounding function to approximate a floor of numerical values for the SQT;

maintaining non-zero gradients of the SQT values by utilizing a differentiable clipping function that computes a soft approximation of the SQT values based on an integer range of a JPEG compression and a scale parameter; and

applying the SQT to DCT patches of the image by rounding a result from dividing DCT coefficients with a corresponding value in the SQT;

encode the quantized DCT coefficients using lossless entropy coding to generate a compressed JPEG file, and decode the compressed JPEG file by reversing the lossless entropy coding to obtain the quantized DCT coefficients;

dequantize the quantized DCT coefficients using the scaled quantization table to generate dequantized DCT coefficients to restore the precision;

apply an inverse discrete cosine transform (IDCT) to the dequantized DCT coefficients to convert the dequantized DCT coefficients back to a spatial domain; and

upsample the chrominance channels to an original resolution, and converting the YCbCr image back to the RGB color space.

9 . The system of claim 8 , wherein the instructions for converting the RGB image to the YCbCr color space comprise instructions for applying a pixel-wise affine transformation.

10 . The system of claim 8 , wherein the instructions for chroma subsampling include instructions for performing anti-aliasing using a 2D convolution followed by downsampling.

11 . The system of claim 8 , wherein a scale factor for the quantization table is computed based on a JPEG quality parameter, and the instructions for scaling the quantization table include instructions for computing the scale factor based on the JPEG quality parameter.

12 . The system of claim 8 , wherein the instructions for encoding the quantized DCT coefficients using lossless entropy coding include instructions for arranging the coefficients in a zigzag order, performing run-length encoding, and applying Huffman coding.

13 . The system of claim 8 , wherein the instructions for the decoding further include instructions for clipping output pixel values to ensure they remain within a valid range by utilizing a product of the output pixel values and the scale parameter when the pixel values are outside the valid range.

14 . The system of claim 8 , wherein the instructions further include using a straight-through estimator (STE) technique during the encoding and decoding to execute accurate gradient propagation for machine learning applications.

15 . A computer program product for encoding and decoding images using a differentiable JPEG approach, the computer program product comprising a non-transitory computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a hardware processor to cause the hardware processor to:

convert an image from a Red-Green-Blue (RGB) color space to a Luminance-Chrominance (YCbCr) color space to obtain a luminance channel and chrominance channels, and apply chroma subsampling to the chrominance channels to reduce resolution;

divide the YCbCr image into pixel blocks and performing a discrete cosine transform (DCT) on each of the pixel blocks to obtain DCT coefficients;

quantize the DCT coefficients using a scaled quantization table (SQT) to generate quantized DCT coefficients and reduce precision by:

computing SQT values with a differentiable floor function that utilizes a polynomial rounding function to approximate a floor of numerical values for the SQT;

maintaining non-zero gradients of the SQT values by utilizing a differentiable clipping function that computes a soft approximation of the SQT values based on an integer range of a JPEG compression and a scale parameter; and

applying the SQT to DCT patches of the image by rounding a result from dividing DCT coefficients with a corresponding value in the SQT;

encode the quantized DCT coefficients using lossless entropy coding to generate a compressed JPEG file, and decode the compressed JPEG file by reversing the lossless entropy coding to obtain the quantized DCT coefficients;

dequantize the quantized DCT coefficients using the scaled quantization table to generate dequantized DCT coefficients to restore the precision;

apply an inverse discrete cosine transform (IDCT) to the dequantized DCT coefficients to convert the dequantized DCT coefficients back to a spatial domain; and

upsample the chrominance channels to an original resolution, and converting the YCbCr image back to the RGB color space.

16 . The computer program product of claim 15 , wherein the program instructions for converting the RGB image to the YCbCr color space comprise instructions for applying a pixel-wise affine transformation.

17 . The computer program product of claim 15 , wherein the program instructions for chroma sub sampling include instructions for performing anti-aliasing using a 2D convolution followed by downsampling.

18 . The computer program product of claim 15 , wherein a scale factor for the quantization table is computed based on a JPEG quality parameter, and the program instructions for scaling the quantization table include instructions for computing the scale factor based on the JPEG quality parameter.

19 . The computer program product of claim 15 , wherein the program instructions for encoding the quantized DCT coefficients using lossless entropy coding include instructions for arranging the coefficients in a zigzag order, performing run-length encoding, and applying Huffman coding.

20 . The computer program product of claim 15 , wherein the program instructions for decoding further include instructions for clipping output pixel values to ensure they remain within a valid range by utilizing a product of the output pixel values and the scale parameter when the pixel values are outside the valid range.