Video encoding and decoding based on image refinement
A particular implementation forms an initial reconstructed image block from inverse quantization and inverse transform, and further refines the reconstructed image block using pixels from neighboring reconstructed blocks. The image block may be refined using a bilateral filter, whose space parameter and range parameter are adaptive to the quantization parameter. The particular implementation can be used in both encoding and decoding when reconstructing an image block. When used in encoding, the particular implementation can be used jointly with coefficient truncation, where some non-zero transform coefficients are set to zero. The number of remaining non-zero transform coefficients after coefficient truncation may be adaptive to the quantization parameter, the variance of the image block, the number of non-zero transform coefficients of the image block, and the index of the last non-zero transform coefficient in a zigzag scanning order.
1. A method, comprising:
inverse quantizing transform coefficients of a block in an image to form de-quantized transform coefficients;
inverse transforming the de-quantized transform coefficients to form prediction residuals;
forming a reconstructed image block using the prediction residuals;
filtering, using at least one filter, a set of pixels to form filtered pixels, the set of pixels comprising a plurality of pixels of the reconstructed image block and a plurality of pixels of reconstructed blocks adjacent to the block,
wherein the filtered pixels comprising a first subset of filtered pixels, which are filtered results for pixel locations corresponding to the plurality of pixels of the reconstructed image block and a second subset of filtered pixels, which are filtered results for pixel locations corresponding to the plurality of pixels of the reconstructed blocks adjacent to the block; and
replacing the plurality of pixels of the reconstructed image block with the first subset of filtered pixels to form a refined image block, wherein the refined image block is used to encode or decode another block of the image, and wherein the plurality of pixels of the reconstructed blocks adjacent to the block are not replaced by the second subset of filtered pixels.
2. The method of claim 1 , wherein the reconstructed blocks adjacent to the block are encoded or decoded before the block.
3. The method of claim 1 , wherein the reconstructed image block forms an extended image block, together with pixels that form part of the reconstructed blocks adjacent to the reconstructed image block, and wherein the filtering is performed on the extended image block.
4. The method of claim 1 , wherein the filtering is performed by a bilateral filter.
5. The method of claim 4 , wherein a space parameter and a range parameter of the bilateral filter vary with a quantization parameter.
6. The method of claim 1 , wherein the method is a method of encoding, further comprising:
setting one or more of initial transform coefficients to zero;
quantizing the initial transform coefficients to the transform coefficients, wherein the transform coefficients are to be inverse quantized.
7. The method of claim 6 , wherein the number of remaining non-zero initial transform coefficients after the one or more of the initial transform coefficient are set to zero varies with at least one of a quantization parameter, a variance of the block, the number of non-zero initial coefficients of the block, and an index of the last non-zero initial coefficient in a scanning order.
8. An apparatus, comprising at least a memory and one or more processors, the one or more processors configured to:
inverse quantize transform coefficients of a block in an image to de-quantized transform coefficients;
inverse transform the de-quantized transform coefficients to prediction residuals;
form a reconstructed image block using the prediction residuals; and
filter a set of pixels to form filtered pixels, the set of pixels comprising a plurality of pixels of the reconstructed image block and a plurality of pixels of reconstructed blocks adjacent to the block,
wherein the filtered pixels comprising a first subset of filtered pixels, which are filtered results for pixel locations corresponding to the plurality of pixels of the reconstructed image block and a second subset of filtered pixels, which are filtered results for pixel locations corresponding to the plurality of pixels of the reconstructed blocks adjacent to the block, and
replace the plurality of pixels of the reconstructed image block with the first subset of filtered pixels to form a refined image block, wherein the refined image block is used to encode or decode another block of the image, and wherein the plurality of pixels of the reconstructed blocks adjacent to the block are not replaced by the second subset of filtered pixels.
9. The apparatus of claim 8 , wherein the reconstructed blocks adjacent to the block are encoded or decoded before the block.
10. The apparatus of claim 8 , wherein the reconstructed image block forms an extended image block, together with pixels that form part of the reconstructed blocks adjacent to the reconstructed image block, and wherein the one or more processors are configured to perform filtering on the extended image block.
11. The apparatus of claim 8 , wherein a bilateral filter is used for filtering.
12. The apparatus of claim 11 , wherein the bilateral filter varies a space and a range parameter with a quantization parameter.
13. The apparatus of claim 8 , wherein the apparatus is an encoder and wherein one or more of initial transform coefficients are set to zero before being quantized into the transform coefficients, the transform coefficients then being inverse quantized, and wherein the number of remaining non-zero initial transform coefficients after the one or more of the initial transform coefficient are set to zero varies with at least one of a quantization parameter, a variance of the block, the number of non-zero initial coefficients of the block, and an index of the last non-zero initial coefficient in a scanning order.
14. A non-transitory processor readable medium having stored thereupon instructions for causing one or more processors to collectively perform:
inverse quantizing transform coefficients of a block in an image to form de-quantized transform coefficients;
inverse transforming the de-quantized transform coefficients to form prediction residuals;
forming a reconstructed image block using the prediction residuals;
filtering a set of pixels to form filtered pixels, the set of pixels comprising a plurality of pixels of the reconstructed image block and a plurality of pixels of reconstructed blocks adjacent to the block, wherein the filtered pixels comprising a first subset of filtered pixels, which are filtered results for pixel locations corresponding to the plurality of pixels of the reconstructed image block and a second subset of filtered pixels, which are filtered results for pixel locations corresponding to the plurality of pixels of the reconstructed blocks adjacent to the block; and
replacing the plurality of pixels of the reconstructed image block with the first subset of filtered pixels to form a refined image block, wherein the refined image block is used to encode or decode another block of the image, and wherein the plurality of pixels of the reconstructed blocks adjacent to the block are not replaced by the second subset of filtered pixels.