Image compression based on parameter-assisted inpainting
View Patent ↗Systems and methods provide image compression based on parameter-assisted inpainting. In one implementation of an encoder, an image is partitioned into blocks and the blocks classified as smooth or unsmooth, based on the degree of visual edge content and chromatic variation in each block. Image content of the unsmooth blocks is compressed, while image content of the smooth blocks is summarized by parameters, but not compressed. The parameters, once obtained, may also be compressed. At a decoder, the compressed image content of the unsmooth blocks and the compressed parameters of the smooth blocks are each decompressed. Each smooth block is then reconstructed by inpainting, guided by the parameters in order to impart visual detail from the original image that cannot be implied from the image content of neighboring blocks that have been decoded.
1. A method, comprising:
under control of one or more processors configured with executable instructions:
partitioning an image into blocks;
determining whether each block is visually smooth;
associating parameters with each visually smooth block, wherein the parameters assist an inpainting process to reconstruct the smooth block;
compressing the image content of each visually unsmooth block; and
compressing the parameters of each visually smooth block.
2. The method as recited in claim 1 , wherein determining whether each block is visually smooth includes determining that the block is visually unsmooth if the block includes visual edge pixels beyond a threshold.
3. The method as recited in claim 1 , wherein determining whether each block is visually smooth includes determining that the block is visually unsmooth if the block includes a variation in chrominance beyond a threshold.
4. The method as recited in claim 1 , wherein the parameters include a parameter selected from the group of parameters consisting of: a positional parameter describing the placement of the block in the image, a luminance parameter describing a variation in luminance in the block, a chrominance parameter describing a variation in chrominance in the block, and a directional parameter describing a placement of a variation in the luminance or the chrominance.
5. The method as recited in claim 1 , further comprising compressing the image content of the visually unsmooth blocks with a JPEG compression and compressing the parameters of the visually smooth blocks with a JBIG compression.
6. The method as recited in claim 1 , further comprising sending the compressed parameters associated with the smooth blocks and sending the compressed image content associated with the unsmooth blocks to a decoder.
7. The method as recited in claim 1 , further comprising:
receiving the compressed parameters and the compressed image content;
decompressing the compressed image content into visually unsmooth blocks of the image;
decompressing the parameters; and
inpainting to restore each smooth block of the image based at least in part on the parameters.
8. The method as recited in claim 7 , further comprising establishing an inpainting priority of the visually smooth blocks, wherein:
a smooth block that has the most neighboring blocks with decompressed image content has the highest priority; and
between two smooth blocks that have the same number of neighboring blocks with decompressed image content, the smooth block that has the most neighboring blocks that are already inpainted has the higher priority.
9. The method as recited in claim 7 , wherein the inpainting further includes inpainting based on visual attributes of the decompressed image content of one or more visually unsmooth blocks neighboring the block being inpainted.
10. The method as recited in claim 7 , wherein the inpainting further includes establishing attributes of border pixels on a border of the block being inpainted based on border pixels of a neighboring block that has decompressed image content along the same border.
11. The method as recited in claim 10 , wherein establishing attributes of border pixels includes bilinearly interpolating an attribute of the border pixels of the block being inpainted with an average of the corresponding attribute of the border pixels of the neighboring block along the same border.
12. The method as recited in claim 10 , wherein the inpainting applies a linear function that is:
continuous in the entire smooth block including the border pixels; and
provides visual change in the smooth block based on the parameters.
13. The method as recited in claim 12 , wherein the function changes depending on how many of the neighboring blocks have decompressed image content.
14. A system comprising:
an image encoder to perform acts comprising:
partitioning an image into blocks;
determining whether each block is visually smooth;
associating parameters with each visually smooth block, wherein the parameters assist an inpainting process to reconstruct the smooth block;
compressing the image content of each visually unsmooth block; and
compressing the parameters of each visually smooth block.
15. The system of claim 14 , wherein determining whether each block is visually smooth includes determining that the block is visually unsmooth if:
the block includes visual edge pixels beyond a threshold, or
the block includes a variation in chrominance beyond a threshold.
16. The system of claim 14 , wherein the parameters include a parameter selected from the group of parameters consisting of: a positional parameter describing the placement of the block in the image, a luminance parameter describing a variation in luminance in the block, a chrominance parameter describing a variation in chrominance in the block, and a directional parameter describing a placement of a variation in the luminance or the chrominance.
17. The system of claim 14 , the image encoder to perform acts further comprising:
compressing the image content of the visually unsmooth blocks with a JPEG compression and compressing the parameters of the visually smooth blocks with a JBIG compression; and
sending the compressed parameters associated with the smooth blocks and sending the compressed image content associated with the unsmooth blocks to a decoder.
18. The system of claim 14 , the image decoder to perform acts comprising:
receiving the compressed parameters and the compressed image content;
decompressing the compressed image content into visually unsmooth blocks of the image;
decompressing the parameters; and
inpainting to restore each smooth block of the image based at least in part on the parameters.
19. The system of claim 18 , the image decoder to perform acts further comprising establishing an inpainting priority of the visually smooth blocks, wherein:
a smooth block that has the most neighboring blocks with decompressed image content has the highest priority; and
between two smooth blocks that have the same number of neighboring blocks with decompressed image content, the smooth block that has the most neighboring blocks that are already inpainted has the higher priority.
20. The system of claim 19 , wherein the inpainting further includes establishing attributes of border pixels on a border of the block being inpainted based on border pixels of a neighboring block that has decompressed image content along the same border, and wherein establishing attributes of border pixels includes bilinearly interpolating an attribute of the border pixels of the block being inpainted with an average of the corresponding attribute of the border pixels of the neighboring block along the same border.