Perceptually coding images and videos

1. A method for encoding a sequence of images of pixels, wherein each image includes one or more blocks processed in a predetermined order, comprising for each block the steps of:

determining a just-noticeable-distortion (JND) for the block;

performing template matching on the block to select candidate blocks in the corresponding image and weights according to a structural similarity and a perceptual distortion of the block with respect to the candidate blocks, wherein the perceptual distortion is a function of the JND, and wherein the weights are equal and a function of a count of a number of the candidate blocks;

determining a prediction residual, wherein the prediction residual is a difference between the input block and an output of a filter combining the candidate blocks;

transforming the prediction residual to produce a transformed prediction residual;

quantizing the transformed prediction residual according to the JND to produce a quantized prediction residual, and wherein the template matching and the quantizing are optimized jointly using the perceptual distortion; and

entropy encoding the quantized prediction residual and the weights for a bitstream, wherein the steps are performed in an encoder.

2. The method of claim 1 , wherein an average magnitude of the transformed prediction residual is signaled in the bitstream, and the JND is a function of the average magnitude.

3. The method of claim 1 , wherein an average magnitude of the transformed prediction residual is estimated based on previously encoded blocks.

4. The method of claim 1 , wherein the filter is a nonlinear combination of the candidate blocks.

5. The method of claim 1 , wherein the JND is determined from perceived frequency components of the pixels.

6. The method of claim 1 , wherein the JND is determined from perceived intensity components of the pixels.

7. The method of claim 1 , wherein the filter is a median filter.

8. The method of claim 1 , further comprising:

minimizing a rate-distortion of the bitstream using according to a perceptual distortion metric.

9. The method of claim 8 , wherein the perceptual distortion metric is a function of the perceptual distortion and the JND.

10. The method of claim 1 , wherein the JND is a function of motion in the sequence of images.

11. The method of claim 1 , wherein the JND includes measurements based on previously encoded blocks.

12. The method of claim 1 , wherein the perceptual distortion is the structural similarity divided by a weighted supra-threshold distortion metric.

13. The method of claim 1 , wherein the perceptual distortion is the structural similarity added to a weighted supra-threshold distortion metric.

14. The method of claim 1 , wherein the weights are determined as the perceptual distortion for a given candidate block divided by a sum of perceptual distortions of all the candidate blocks.

15. The method of claim 1 , further comprising:

wherein the encoding is according to a High Efficiency Video Coding (HEVC) standard, and wherein the HEVC standard includes a set of prediction modes, and an additional prediction mode for the set of predictions is the output of the filter.

16. The method of claim 15 , wherein a distortion resulting from a particular prediction mode is weighted by a function of the JND.

17. The method of claim 1 , further comprising:

entropy decoding the bitstream to obtain the quantized prediction residual, and the weights;

performing an inverse quantizing of the quantized prediction residual to obtain a reconstructed prediction residual;

performing the template matching on previously decoded blocks to obtain the candidate blocks;

determining a prediction corresponding to the output of the filter;

combining the reconstructed prediction residual and the prediction to obtain a decoded block.

18. The method of claim 17 , wherein an average magnitude is decoded from the bitstream, and the JND is a function of the average magnitude.