Image compression using tensor-product B-spline representation
Methods and apparatus for image compression/decompression using tensor product B-spline (TPB) representations. According to an example embodiment, an image-compression method includes generating a plurality of TPB models representing an image, including a TPB model for approximating a spatial reshaping function and a plurality of patch-wise TPB models for estimating the image signal. In some examples, the plurality of patch-wise TPB models includes a plurality of luma-channel TPB models and a plurality of chroma-channel TPB models. The spatial reshaping function is configured to shift a non-uniform distribution of local content complexity in the image toward being more uniform. The method also includes generating a metadata stream carrying metadata representing sets of coefficients of various TPB models. At least a portion of the metadata is generated by quantizing the corresponding coefficients to a smaller number of bits and applying arithmetic coding to the quantized coefficients.
1 . An image-compression method, comprising:
generating, with an electronic encoder, a first tensor product B-spline “TPB” model representing an image, the first TPB model approximating a spatial domain coordinate reshaping function configured to shift a non-uniform distribution of local content complexity in the image toward a more uniform distribution of local content complexity;
generating, with the electronic encoder, a plurality of second TPB models, each of the second TPB models representing a respective patch of the image, wherein each of the second TPB models provide an estimate of an image signal within the respective patch as a function of reshaped coordinate, the reshaped coordinate being determined using the spatial domain coordinate reshaping function; and
generating, with the electronic encoder, a metadata stream including first metadata representing a set of coefficients of the first TPB model and second metadata representing sets of coefficients of the plurality of second TPB models.
2 . The image-compression method of claim 1 , wherein the reshaped coordinate is normalized to be in a range [0 1].
3 . The image-compression method of claim 1 ,
wherein generating the second metadata comprises compressing, with the electronic encoder, the sets of coefficients of the plurality of second TPB models.
4 . The image-compression method of claim 3 , wherein said compressing comprises:
generating quantized sets of coefficients by quantizing each coefficient to a smaller number of bits; and
applying entropy coding to the quantized sets of coefficients.
5 . The image-compression method of claim 4 ,
wherein the entropy coding comprises arithmetic coding; and
wherein the metadata stream includes side information characterizing at least one of the quantizing and the arithmetic coding.
6 . The image-compression method of claim 4 , wherein said quantizing comprises applying nonlinear quantization to the coefficients of the plurality of second TPB models.
7 . The image-compression method of claim 1 , further comprising partitioning the image, with the electronic encoder, into a plurality of nonoverlapping patches,
wherein each respective patch is selected from the plurality of nonoverlapping patches.
8 . The image-compression method of claim 1 ,
wherein the image is a YCbCr image; and
wherein the method further comprises generating the YCbCr image by applying RGB-to-YCbCr conversion to an RGB image.
9 . The image-compression method of claim 1 , wherein the plurality of second TPB models includes a plurality of luma-channel TPB models and a plurality of chroma-channel TPB models.
10 . The image-compression method of claim 1 ,
wherein a two-dimensional standard deviation is a measure of the local content complexity.
11 . The image-compression method of claim 1 ,
wherein said generating the plurality of second TPB models comprises, for a second TPB model from the plurality of second TPB models, finding a respective set of coefficients that causes a difference between the image signal and an estimate of the image signal in the respective patch to be approximately minimized, the estimate of the image signal being computed using the second TPB model.
12 . The image-compression method of claim 11 , wherein said finding is based on least squares minimization.
13 . The image-compression method of claim 1 , further comprising transmitting the metadata stream, via a communication channel, to an electronic decoder.
14 . The image-compression method of claim 1 , further comprising performing rate-distortion optimization to select one or more parameters from the group consisting of a patch size, a number of basis functions, and a number of bits for a quantized TPB coefficient.
15 . The image-compression method of claim 14 , wherein the rate-distortion optimization is based on a greedy algorithm.
16 . The image-compression method of claim 1 , wherein the spatial reshaping function is a normalized cumulative function of local standard deviation computed via a process of standard deviation equalization.
17 . A non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, cause the electronic processor to perform operations comprising the method of claim 1 .
18 . An apparatus for image compression, the apparatus comprising:
at least one processor; and
at least one memory including program code, wherein the at least one memory and the program code are configured to, with the at least one processor, cause the apparatus at least to:
generate a first tensor product B-spline (TPB) model representing an image, the first TPB model approximating a spatial domain coordinate reshaping function configured shift a non-uniform distribution of local content complexity in the image toward a more uniform distribution of local content complexity;
generate a plurality of second TPB models, each of the second TPB models representing a respective patch of the image, wherein each of the second TPB models provide an estimate of an image signal within the respective patch as a function of reshaped coordinate, the reshaped coordinate being determined using the spatial domain coordinate reshaping function; and
generate a metadata stream including first metadata representing a set of coefficients of the first TPB model and second metadata representing sets of coefficients of the plurality of second TPB models.
19 . The apparatus of claim 18 , wherein the reshaped coordinate is normalized to be in a range [0 1].
20 . An image-decompression method, comprising:
receiving, with an electronic decoder, a metadata stream including first metadata representing a set of coefficients of a first tensor product B-spline “TPB” model and second metadata representing sets of coefficients of a plurality of second TPB models, the first TPB model approximating a spatial domain coordinate reshaping function configured to shift a non-uniform distribution of local content complexity in a source image toward a more uniform distribution of local content complexity, each of the second TPB models representing a respective patch of the source image and providing an estimate of an image signal within the respective patch as a function of reshaped coordinate, the reshaped coordinate being determined based on the spatial domain coordinate reshaping function;
generating, with the electronic decoder, a plurality of reconstructed image patches based on the first TPB model and further based on the plurality of second TPB models, the first TPB model being reconstructed using the first metadata, the plurality of second TPB models being reconstructed using the second metadata; and
constructing, with the electronic decoder, an estimated image by assembling the plurality of reconstructed image patches in an image frame.
21 . The image-decompression method of claim 20 , wherein the reshaped coordinate is normalized to be in a range [0 1].
22 . The image-decompression method of claim 20 , further comprising decompressing the second metadata, with the electronic decoder, to reconstruct the sets of coefficients of the plurality of second TPB models.
23 . The image-decompression method of claim 22 , wherein said decompressing is based on side information included with the metadata stream, the side information characterizing at least one of quantization and arithmetic coding of the sets of coefficients of the plurality of second TPB models.
24 . The image-decompression method of claim 20 , wherein the plurality of second TPB models includes a plurality of luma-channel TPB models and a plurality of chroma-channel TPB models.
25 . The image-decompression method of claim 20 , wherein a two-dimensional standard deviation is a measure of the local content complexity.
26 . A non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, cause the electronic processor to perform operations comprising the method of claim 20 .
27 . An apparatus for image decompression, the apparatus comprising:
at least one processor; and
at least one memory including program code,
wherein the at least one memory and the program code are configured to, with the at least one processor, cause the apparatus at least to:
receive a metadata stream including first metadata representing a set of coefficients of a first tensor product B-spline (TPB) model and second metadata representing sets of coefficients of a plurality of second TPB models, the first TPB model approximating a spatial domain coordinate reshaping function configured to shift a non-uniform distribution of local content complexity in a source image toward a more uniform distribution of local content complexity, each of the second TPB models representing a respective patch of the source image and providing an estimate of an image signal within the respective patch as a function of reshaped coordinate, the reshaped coordinate being determined using the spatial domain coordinate reshaping function;
generate a plurality of reconstructed image patches based on the first TPB model and further based on the plurality of second TPB models, the first TPB model being reconstructed using the first metadata, the plurality of second TPB models being reconstructed using the second metadata; and
construct an estimated image by assembling the plurality of reconstructed image patches in an image frame.
28 . The apparatus of claim 27 , wherein the reshaped coordinate is normalized to be in a range [0 1].