Video coding/decoding with sub-block transform sizes and adaptive deblock filtering
Techniques and tools for video coding/decoding with motion resolution switching and sub-block transform coding/decoding are described. For example, a video encoder adaptively switches the resolution of motion estimation and compensation between quarter-pixel and half-pixel resolutions; a corresponding video decoder adaptively switches the resolution of motion compensation between quarter-pixel and half-pixel resolutions. For sub-block transform sizes, for example, a video encoder adaptively switches between 8×8, 8×4, and 4×8 DCTs when encoding 8×8 prediction residual blocks; a corresponding video decoder switches between 8×8, 8×4, and 4×8 inverse DCTs during decoding.
1. In a computer system, a method comprising:
reconstructing a frame of a video sequence;
adaptively filtering one or more boundaries between multiple blocks in the reconstructed frame to reduce boundary discontinuities, wherein application of the filtering across a given boundary of the one or more boundaries includes:
computing a cross boundary discontinuity measure that quantifies pixel value discontinuity across the given boundary, wherein the cross boundary discontinuity measure is computed using at least one pixel value on a first side of the given boundary and at least one pixel value on a second side of the given boundary, the first side of the given boundary being opposite the second side of the given boundary;
computing a first side discontinuity measure that quantifies pixel value discontinuity on the first side of the given boundary, wherein the first side discontinuity measure is computed using multiple pixel values on the first side of the given boundary;
computing a second side discontinuity measure that quantifies pixel value discontinuity on the second side of the given boundary, wherein the second side discontinuity measure is computed using multiple pixel values on the second side of the given boundary; and
determining whether to filter the given boundary and, if so, filtering the given boundary, including adjusting the filtering the given boundary depending at least in part on the cross boundary discontinuity measure, the first side discontinuity measure, and the second side discontinuity measure; and
storing the adaptively filtered, reconstructed frame in a frame buffer for use as a reference frame in subsequent motion compensation.
2. The method of claim 1 , wherein the reconstructing the frame includes, for each of the multiple blocks of the frame:
reconstructing a prediction residual for the block using a variable-block-size inverse frequency transform;
computing a motion-compensated prediction for the block using one or more motion vectors to predict pixel values of the block; and
reconstructing the block based upon the prediction residual and the motion-compensated prediction.
3. The method of claim 1 , further comprising reconstructing a previous frame of the video sequence, including, for each of multiple blocks of the previous frame:
computing a spatial extrapolation for the block;
reconstructing a spatial-extrapolation residual for the block, including applying an inverse re-oriented frequency transform to the spatial-extrapolation residual; and
combining the spatial-extrapolation residual and the spatial extrapolation.
4. The method of claim 1 , further comprising selectively disabling the filtering for a subsequent frame of the video sequence.
5. The method of claim 1 , wherein the application of the filtering further depends at least in a part upon a quantization level.
6. The method of claim 1 , wherein the one or more boundaries that are adaptively filtered are boundaries between 8×8 blocks, even when the transform size of the variable-block-size inverse frequency transform is smaller than 8×8.
7. The method of claim 1 , wherein the one or more boundaries are in a luminance plane of the reconstructed frame.
8. The method of claim 1 , wherein a single line of pixel values across the given boundary includes the multiple pixel values on the first side and the multiple pixel values on the second side.
9. One or more computer-readable memory or storage devices storing computer-executable instructions for causing a computer system, when programmed thereby, to perform operations comprising:
reconstructing a frame of a video sequence;
adaptively filtering one or more boundaries between multiple blocks in the reconstructed frame to reduce boundary discontinuities, wherein application of the filtering across a given boundary of the one or more boundaries includes:
computing a cross boundary discontinuity measure that quantifies pixel value discontinuity across the given boundary, wherein the cross boundary discontinuity measure is computed using at least one pixel value on a first side of the given boundary and at least one pixel value on a second side of the given boundary, the first side of the given boundary being opposite the second side of the given boundary;
computing a first side discontinuity measure that quantifies pixel value discontinuity on the first side of the given boundary, wherein the first side discontinuity measure is computed using multiple pixel values on the first side of the given boundary;
computing a second side discontinuity measure that quantifies pixel value discontinuity on the second side of the given boundary, wherein the second side discontinuity measure is computed using multiple pixel values on the second side of the given boundary; and
determining whether to filter the given boundary and, if so, filtering the given boundary, including adjusting the filtering the given boundary depending at least in part on the cross boundary discontinuity measure, the first side discontinuity measure, and the second side discontinuity measure; and
storing the adaptively filtered, reconstructed frame in a frame buffer for use as a reference frame in subsequent motion compensation.
10. The one or more computer-readable memory or storage devices of claim 9 , wherein the reconstructing the frame includes, for each of the multiple blocks of the frame:
reconstructing a prediction residual for the block using a variable-block-size inverse frequency transform;
computing a motion-compensated prediction for the block using one or more motion vectors to predict pixel values of the block; and
reconstructing the block based upon the prediction residual and the motion-compensated prediction.
11. The one or more computer-readable memory or storage devices of claim 9 , wherein the operations further comprise reconstructing a previous frame of the video sequence, including, for each of multiple blocks of the previous frame:
computing a spatial extrapolation for the block;
reconstructing a spatial-extrapolation residual for the block, including applying an inverse re-oriented frequency transform to the spatial-extrapolation residual; and
combining the spatial-extrapolation residual and the spatial extrapolation.
12. The one or more computer-readable memory or storage devices of claim 9 , wherein the operations further comprise selectively disabling the filtering for a subsequent frame of the video sequence.
13. The one or more computer-readable memory or storage devices of claim 9 , wherein the application of the filtering further depends at least in a part upon a quantization level.
14. The one or more computer-readable memory or storage devices of claim 9 , wherein the one or more boundaries that are adaptively filtered are boundaries between 8×8 blocks, even when the transform size of the variable-block-size inverse frequency transform is smaller than 8×8.
15. The one or more computer-readable memory or storage devices of claim 9 , wherein the one or more boundaries are in a luminance plane of the reconstructed frame.
16. The one or more computer-readable memory or storage devices of claim 9 , wherein a single line of pixel values across the given boundary includes the multiple pixel values on the first side and the multiple pixel values on the second side.
17. A computer system comprising a processing unit and memory, wherein the computer system is configured to perform operations comprising:
reconstructing a frame of a video sequence;
adaptively filtering one or more boundaries between multiple blocks in the reconstructed frame to reduce boundary discontinuities, wherein application of the filtering across a given boundary of the one or more boundaries includes:
computing a cross boundary discontinuity measure that quantifies pixel value discontinuity across the given boundary, wherein the cross boundary discontinuity measure is computed using at least one pixel value on a first side of the given boundary and at least one pixel value on a second side of the given boundary, the first side of the given boundary being opposite the second side of the given boundary;
computing a first side discontinuity measure that quantifies pixel value discontinuity on the first side of the given boundary, wherein the first side discontinuity measure is computed using multiple pixel values on the first side of the given boundary;
computing a second side discontinuity measure that quantifies pixel value discontinuity on the second side of the given boundary, wherein the second side discontinuity measure is computed using multiple pixel values on the second side of the given boundary; and
determining whether to filter the given boundary and, if so, filtering the given boundary, including adjusting the filtering the given boundary depending at least in part on the cross boundary discontinuity measure, the first side discontinuity measure, and the second side discontinuity measure; and
storing the adaptively filtered, reconstructed frame in a frame buffer for use as a reference frame in subsequent motion compensation.
18. The computer system of claim 17 , wherein the application of the filtering further depends at least in a part upon a quantization level.
19. The computer system of claim 17 , wherein the one or more boundaries that are adaptively filtered are boundaries between 8×8 blocks, even when the transform size of the variable-block-size inverse frequency transform is smaller than 8×8.
20. The computer system of claim 17 , wherein the one or more boundaries are in a luminance plane of the reconstructed frame.