Encoding and decoding images using differentiable JPEG compression
Systems and methods are provided for encoding and decoding images using differentiable JPEG compression, including converting images from RGB color space to YCbCr color space to obtain a luminance and chrominance channels, and applying chroma subsampling to the chrominance channels to reduce resolution. The YCbCr image is divided into pixel blocks and a DCT is performed on the pixel blocks to obtain DCT coefficients. DCT coefficients are quantized using a scaled quantization table to reduce precision, and quantized DCT coefficients are encoded using lossless entropy coding, forming a compressed JPEG file decoded by reversing the lossless entropy coding to obtain quantized DCT coefficients, which are dequantized using the scaled quantization table to restore the precision. The dequantized DCT coefficients are converted back to a spatial domain using an IDCT, the chrominance channels are upsampled to original resolution, and the YCbCr image is converted back to the RGB color space.
1 . A computer-implemented method for encoding and decoding images using differentiable Joint Photographic Experts Group (JPEG) compression, comprising:
converting an image from a Red-Green-Blue (RGB) color space to a Luminance-Chrominance (YCbCr) color space to obtain a luminance channel and chrominance channels, and applying chroma subsampling to the chrominance channels to reduce resolution;
dividing the YCbCr image into pixel blocks and performing a discrete cosine transform (DCT) on each of the pixel blocks to obtain DCT coefficients;
quantizing the DCT coefficients using a scaled quantization table (SQT) to generate quantized DCT coefficients and reduce precision by:
computing SQT values with a differentiable floor function that utilizes a polynomial rounding function to approximate a floor of numerical values for the SQT;
maintaining non-zero gradients of the SQT values by utilizing a differentiable clipping function that computes a soft approximation of the SQT values based on an integer range of the JPEG compression and a scale parameter; and
applying the SQT to DCT patches of the image by rounding a result from dividing DCT coefficients with a corresponding value in the SQT;
encoding the quantized DCT coefficients using lossless entropy coding to generate a compressed JPEG file and decoding the compressed JPEG file by reversing the lossless entropy coding to obtain the quantized DCT coefficients;
dequantizing the quantized DCT coefficients using the scaled quantization table to generate dequantized DCT coefficients to restore the precision;
applying an inverse discrete cosine transform (IDCT) to the dequantized DCT coefficients to convert the dequantized DCT coefficients back to a spatial domain; and
upsampling the chrominance channels to an original resolution, and converting the YCbCr image back to the RGB color space.
2 . The method of claim 1 , wherein converting the RGB image to the YCbCr color space comprises applying a pixel-wise affine transformation.
3 . The method of claim 1 , wherein the chroma subsampling involves anti-aliasing using a 2D convolution followed by downsampling.
4 . The method of claim 1 , wherein a scale factor for the quantization table is computed based on a JPEG quality parameter, and the quantization table is scaled by the scale factor.
5 . The method of claim 1 , wherein the encoding the quantized DCT coefficients using lossless entropy coding comprises arranging the coefficients in a zigzag order, performing run-length encoding, and applying Huffman coding.
6 . The method of claim 1 , wherein the decoding further comprises clipping output pixel values to ensure they remain within a valid range by utilizing a product of the output pixel values and the scale parameter when the pixel values are outside the valid range.
7 . The method of claim 1 , further comprising using a straight-through estimator (STE) technique during the encoding and the decoding to execute accurate gradient propagation for machine learning applications.
8 . A system for encoding and decoding images using a differentiable JPEG approach, comprising:
a processor device; and
a memory storing instructions that, when executed by the processor device, cause the system to:
convert an image from a Red-Green-Blue (RGB) color space to a Luminance-Chrominance (YCbCr) color space to obtain a luminance channel and chrominance channels, and apply chroma subsampling to the chrominance channels to reduce resolution;
divide the YCbCr image into pixel blocks and performing a discrete cosine transform (DCT) on each of the pixel blocks to obtain DCT coefficients;
quantize the DCT coefficients using a scaled quantization table (SQT) to generate quantized DCT coefficients and reduce precision by:
computing SQT values with a differentiable floor function that utilizes a polynomial rounding function to approximate a floor of numerical values for the SQT;
maintaining non-zero gradients of the SQT values by utilizing a differentiable clipping function that computes a soft approximation of the SQT values based on an integer range of a JPEG compression and a scale parameter; and
applying the SQT to DCT patches of the image by rounding a result from dividing DCT coefficients with a corresponding value in the SQT;
encode the quantized DCT coefficients using lossless entropy coding to generate a compressed JPEG file, and decode the compressed JPEG file by reversing the lossless entropy coding to obtain the quantized DCT coefficients;
dequantize the quantized DCT coefficients using the scaled quantization table to generate dequantized DCT coefficients to restore the precision;
apply an inverse discrete cosine transform (IDCT) to the dequantized DCT coefficients to convert the dequantized DCT coefficients back to a spatial domain; and
upsample the chrominance channels to an original resolution, and converting the YCbCr image back to the RGB color space.
9 . The system of claim 8 , wherein the instructions for converting the RGB image to the YCbCr color space comprise instructions for applying a pixel-wise affine transformation.
10 . The system of claim 8 , wherein the instructions for chroma subsampling include instructions for performing anti-aliasing using a 2D convolution followed by downsampling.
11 . The system of claim 8 , wherein a scale factor for the quantization table is computed based on a JPEG quality parameter, and the instructions for scaling the quantization table include instructions for computing the scale factor based on the JPEG quality parameter.
12 . The system of claim 8 , wherein the instructions for encoding the quantized DCT coefficients using lossless entropy coding include instructions for arranging the coefficients in a zigzag order, performing run-length encoding, and applying Huffman coding.
13 . The system of claim 8 , wherein the instructions for the decoding further include instructions for clipping output pixel values to ensure they remain within a valid range by utilizing a product of the output pixel values and the scale parameter when the pixel values are outside the valid range.
14 . The system of claim 8 , wherein the instructions further include using a straight-through estimator (STE) technique during the encoding and decoding to execute accurate gradient propagation for machine learning applications.
15 . A computer program product for encoding and decoding images using a differentiable JPEG approach, the computer program product comprising a non-transitory computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a hardware processor to cause the hardware processor to:
convert an image from a Red-Green-Blue (RGB) color space to a Luminance-Chrominance (YCbCr) color space to obtain a luminance channel and chrominance channels, and apply chroma subsampling to the chrominance channels to reduce resolution;
divide the YCbCr image into pixel blocks and performing a discrete cosine transform (DCT) on each of the pixel blocks to obtain DCT coefficients;
quantize the DCT coefficients using a scaled quantization table (SQT) to generate quantized DCT coefficients and reduce precision by:
computing SQT values with a differentiable floor function that utilizes a polynomial rounding function to approximate a floor of numerical values for the SQT;
maintaining non-zero gradients of the SQT values by utilizing a differentiable clipping function that computes a soft approximation of the SQT values based on an integer range of a JPEG compression and a scale parameter; and
applying the SQT to DCT patches of the image by rounding a result from dividing DCT coefficients with a corresponding value in the SQT;
encode the quantized DCT coefficients using lossless entropy coding to generate a compressed JPEG file, and decode the compressed JPEG file by reversing the lossless entropy coding to obtain the quantized DCT coefficients;
dequantize the quantized DCT coefficients using the scaled quantization table to generate dequantized DCT coefficients to restore the precision;
apply an inverse discrete cosine transform (IDCT) to the dequantized DCT coefficients to convert the dequantized DCT coefficients back to a spatial domain; and
upsample the chrominance channels to an original resolution, and converting the YCbCr image back to the RGB color space.
16 . The computer program product of claim 15 , wherein the program instructions for converting the RGB image to the YCbCr color space comprise instructions for applying a pixel-wise affine transformation.
17 . The computer program product of claim 15 , wherein the program instructions for chroma sub sampling include instructions for performing anti-aliasing using a 2D convolution followed by downsampling.
18 . The computer program product of claim 15 , wherein a scale factor for the quantization table is computed based on a JPEG quality parameter, and the program instructions for scaling the quantization table include instructions for computing the scale factor based on the JPEG quality parameter.
19 . The computer program product of claim 15 , wherein the program instructions for encoding the quantized DCT coefficients using lossless entropy coding include instructions for arranging the coefficients in a zigzag order, performing run-length encoding, and applying Huffman coding.
20 . The computer program product of claim 15 , wherein the program instructions for decoding further include instructions for clipping output pixel values to ensure they remain within a valid range by utilizing a product of the output pixel values and the scale parameter when the pixel values are outside the valid range.