Pixel adaptive intra smoothing
The present invention relates to spatial prediction of pixels of a block, the block being a block of a digital image. In particular, for a block pixel at least one reference pixel(s) is selected and out of the selected reference pixel(s), the block pixel is predicted. In particular, the prediction of the block pixel is performed depending on the distance of this pixel to the reference pixel(s) from which it is to be predicted.
1. A video encoding method for use in a video encoder, the video encoding method encoding pixels of an image block in a digital image, the video encoding method comprising:
predicting, using a processor, the pixels of the image block based on reference pixels spatially adjacent to the image block;
encoding a difference between the predicted pixels of the image block and an original image block; and
embedding the encoded difference within a bitstream,
wherein said predicting the pixels of the image block includes performing, for each of the pixels of the image block:
determining, for the pixel of the image block, at least one reference pixel adjacent to the image block;
determining a spatial distance between the pixel of the image block and the at least one reference pixel, the spatial distance being determined by adding (i) a squared absolute value of a vertical distance between the pixel of the image block and the at least one reference pixel and (ii) a squared absolute value of a horizontal distance between the pixel of the image block and the at least one reference pixel; and
predicting the pixel of the image block,
wherein in said predicting the pixel of the image block:
when the determined spatial distance is greater than or equal to a first distance, the pixel is predicted by smoothing the at least one reference pixel with a low-pass filter and using the smoothed at least one reference pixel; and
when the determined spatial distance is smaller than a second distance that is smaller than the first distance, the pixel is predicted using the at least one reference pixel without smoothing the at least one reference pixel with the low-pass filter; and
when the determined spatial distance is smaller than the first distance and greater than or equal to the second distance, the pixel is predicted as a weighted sum of (i) information obtained by predicting the pixel using the at least one reference pixel without smoothing the at least one reference pixel with the low-pass filter and (ii) information obtained by predicting the pixel by smoothing the at least one reference pixel with a low-pass filter and using the smoothed at least one reference pixel.
2. The video encoding method according to claim 1 , wherein in said predicting the pixel of the image block, when the determined spatial distance is greater than or equal to the first distance, the pixel is predicted by smoothing the at least one reference pixel with the low-pass filter, interpolating at least one sub-pel position from the smoothed at least one reference pixel, and extrapolating the at least one pixel of the image block by copying the at least one interpolated sub-pel position.
3. The video encoding method according to claim 1 , wherein weights for the weighted sum are defined for each vertical and/or horizontal component of the spatial distance between the at least one pixel of the image block and the reference pixel used for the prediction of the at least one pixel of the image block.
4. The video encoding method according to claim 3 , wherein the weights are organized in a matrix, such that elements of the matrix are proportional to elements of an autocorrelation matrix of pixels of the image block, such that the elements of the matrix are derived based on statistics of images from a training set, or such that the elements of the matrix are either 0 or 1.
5. The video encoding method according to claim 1 , wherein said predicting predicts a pixel of the image block from at least one sub-pel position obtained by interpolating at least one reference pixel selected for said predicting of the pixel of the image block, and a type of interpolation filtering is different for at least two pixels in the image block.
6. The video encoding method according to claim 5 , wherein the type of interpolation filtering includes at least one of linear filtering, copying the at least one reference pixel, cubic filtering, or Wiener filtering.
7. The video encoding method according to claim 1 , wherein said embedding further includes embedding into the bitstream at least one parameter indicating which filter is to be applied for each pixel of the image block, the at least one parameter being:
a threshold for indicating a distance between a pixel of the image block and a reference pixel used for the prediction of the pixel of the image block up to which a first filtering is applied and starting from which a second filtering is applied, the second filtering being different from the first filtering;
a plurality of weights for combining prediction using reference pixels filtered according to the first filtering and prediction using reference pixels filtered according to the second filtering, the weights being different for at least two different pixel positions within the block; or
for each pixel of the image block, an indicator indicating which filtering is to be applied to the pixel.
8. A video decoding for use in a video decoder, the video decoding method decoding pixels of an image block from a bitstream, the video decoding method comprising:
extracting a block of pixel differences from the bitstream;
predicting, using a processor, the pixels of the image block based on reference pixels spatially adjacent to the image block; and
decoding the pixels of the image block by adding the predicted pixels of the image block of pixels to the extracted block of pixel differences,
wherein said predicting the pixels of the image block includes performing, for each of the pixels of the image block:
determining, for the pixel of the image block, at least one reference pixel adjacent to the image block;
determining a spatial distance between the pixel of the image block and the at least one reference pixel, the spatial distance being determined by adding (i) a squared absolute value of a vertical distance between the pixel of the image block and the at least one reference pixel and (ii) a squared absolute value of a horizontal distance between the pixel of the image block and the at least one reference pixel; and
predicting the pixel of the image block,
wherein in said predicting the pixel of the image block:
when the determined spatial distance is greater than or equal to a first distance, the pixel is predicted by smoothing the at least one reference pixel with a low-pass filter and using the smoothed at least one reference pixel; and
when the determined spatial distance is smaller than a second distance that is smaller than the first distance, the pixel is predicted using the at least one reference pixel without smoothing the at least one reference pixel with the low-pass filter; and
when the determined spatial distance is smaller than the first distance and greater than or equal to the second distance, the pixel is predicted as a weighted sum of (i) information obtained by predicting the pixel using the at least one reference pixel without smoothing the at least one reference pixel with the low-pass filter and (ii) information obtained by predicting the pixel by smoothing the at least one reference pixel with a low-pass filter and using the smoothed at least one reference pixel.
9. A video encoding apparatus for encoding pixels of an image block in a digital image, the video encoding apparatus comprising:
a processor;
a predictor for, using a processor, predicting the pixels of the image block based on reference pixels spatially adjacent to the image block;
an encoder for encoding a difference between the predicted pixels of the image block and an original image block; and
an embedder for embedding the encoded difference within a bitstream,
wherein said predictor predicts the pixels of the image block by performing, for each of the pixels of the image block:
determining, for the pixel of the image block, at least one reference pixel adjacent to the image block, using the processor;
determining a spatial distance between the pixel of the image block and the at least one reference pixel, the spatial distance being determined by adding (i) a squared absolute value of a vertical distance between the pixel of the image block and the at least one reference pixel and (ii) a squared absolute value of a horizontal distance between the pixel of the image block and the at least one reference pixel; and
predicting the pixel of the image block,
wherein when the determined spatial distance is greater than or equal to a first distance, the at least one reference pixel is smoothed with a low-pass filter and said predictor predicts the pixel using the smoothed at least one reference pixel,
when the determined spatial distance is smaller than a second distance that is smaller than the first distance, said predictor predicts the pixel using the at least one reference pixel without smoothing the at least one reference pixel with the low-pass filter, and
when the determined spatial distance is smaller than the first distance and greater than or equal to the second distance, said predictor predicts the pixel using a weighted sum of (i) information obtained by predicting the pixel using the at least one reference pixel without smoothing the at least one reference pixel with the low-pass filter and (ii) information obtained by predicting the pixel by smoothing the at least one reference pixel with a low-pass filter and using the smoothed at least one reference pixel.
10. The video encoding apparatus according to claim 9 , wherein when the determined spatial distance is greater than or equal to a first distance, the at least one reference pixel is smoothed with the low-pass filter, the predictor interpolates at least one sub-pel position from the smoothed at least one reference pixel smoothed, the predictor extrapolates the at least one pixel of the image block by copying the at least one interpolated sub-pel position.
11. The video encoding apparatus according to claim 9 , wherein weights for the weighted sum are defined for each vertical and/or horizontal component of the spatial distance between the at least one pixel of the image block and the reference pixel used for the prediction of the at least one pixel of the image block.
12. The video encoding apparatus according to claim 11 , wherein the weights are organized in a matrix, such that elements of the matrix are proportional to elements of an autocorrelation matrix of pixels of the image block, such that the elements of the matrix are derived based on statistics of images from a training set, or such that the elements of the matrix are either 0 or 1.
13. The video encoding apparatus according to claim 9 , wherein said predictor is configured to predict a pixel of the image block from at least one sub-pel position obtained by interpolating at least one reference pixel selected for the prediction of the pixel of the image block, and a type of interpolation filtering is different for at least two pixels in the image block.
14. The video encoding apparatus according to claim 13 , wherein the type of interpolation filtering includes at least one of linear filtering, copying the at least one reference pixel, cubic filtering, or Wiener filtering.
15. The video encoding apparatus according to claim 9 , wherein said embedder is further configured to embed into the bitstream at least one parameter indicating which filter is to be applied for each pixel of the image block, the at least one parameter being:
a threshold for indicating a distance between a pixel of the image block and a reference pixel used for the prediction of the pixel of the image block up to which a first filtering is applied and starting from which a second filtering is applied, the second filtering being different from the first filtering;
a plurality of weights for combining prediction using reference pixels filtered according to the first filtering and prediction using reference pixels filtered according to the second filtering, the weights being different for at least two different pixel positions within the block; or
for each pixel of the image block, an indicator indicating which filtering is to be applied to the pixel.
16. A video decoding apparatus for decoding pixels of an image block from a bitstream, the video decoding apparatus comprising:
a processor;
an extractor for extracting a block of pixel differences from the bitstream;
a predictor for, using a processor, predicting the pixels of the image block based on reference pixels spatially adjacent to the image block;
a decoder for decoding the pixels of the image block by adding the predicted pixels of the image block to the extracted block of pixel differences,
wherein said predictor predicts the pixels of the image block by performing, for each of the pixels of the image block:
determining, for the pixel of the image block, at least one reference pixel adjacent to the image block, using the processor;
determining a spatial distance between the pixel of the image block and the at least one reference pixel, the spatial distance being determined by adding (i) a squared absolute value of a vertical distance between the pixel of the image block and the at least one reference pixel and (ii) a squared absolute value of a horizontal distance between the pixel of the image block and the at least one reference pixel; and
predicting the pixel of the image block,
wherein when the determined spatial distance is greater than or equal to a first distance, the at least one reference pixel is smoothed with a low-pass filter and said predictor predicts the pixel using the smoothed at least one reference pixel,
when the determined spatial distance is smaller than a second distance that is smaller than the first distance, said predictor predicts the pixel using the at least one reference pixel without smoothing the at least one reference pixel with the low-pass filter, and
when the determined spatial distance is smaller than the first distance and greater than or equal to the second distance, said predictor predicts the pixel using a weighted sum of (i) information obtained by predicting the pixel using the at least one reference pixel without smoothing the at least one reference pixel with the low-pass filter and (ii) information obtained by predicting the pixel by smoothing the at least one reference pixel with a low-pass filter and using the smoothed at least one reference pixel.