Video coding/decoding with sub-block transform sizes and adaptive deblock filtering
Techniques and tools for video coding/decoding with motion resolution switching and sub-block transform coding/decoding are described. For example, a video encoder adaptively switches the resolution of motion estimation and compensation between quarter-pixel and half-pixel resolutions; a corresponding video decoder adaptively switches the resolution of motion compensation between quarter-pixel and half-pixel resolutions. For sub-block transform sizes, for example, a video encoder adaptively switches between 8×8, 8×4, and 4×8 DCTs when encoding 8×8 prediction residual blocks; a corresponding video decoder switches between 8×8, 8×4, and 4×8 inverse DCTs during decoding.
1. A computer system comprising one or more processing units and memory, wherein the computer system implements a video encoder system comprising:
a frame buffer configured to store a reference frame for use in motion compensation;
a video encoder, implemented with the one or more processing units and memory, configured to perform operations to encode video, thereby producing encoded data, the operations including:
reconstructing a frame of a video sequence;
adaptively filtering one or more boundaries between multiple blocks in the reconstructed frame to reduce boundary discontinuities, wherein application of the filtering across a given boundary of the one or more boundaries includes:
computing a cross boundary discontinuity measure that quantifies pixel value discontinuity across the given boundary, wherein the cross boundary discontinuity measure is computed using at least one pixel value on a first side of the given boundary and at least one pixel value on a second side of the given boundary, the first side of the given boundary being opposite the second side of the given boundary;
computing a first side discontinuity measure that quantifies pixel value discontinuity on the first side of the given boundary, wherein the first side discontinuity measure is computed using multiple pixel values on the first side of the given boundary;
computing a second side discontinuity measure that quantifies pixel value discontinuity on the second side of the given boundary, wherein the second side discontinuity measure is computed using multiple pixel values on the second side of the given boundary; and
determining whether to filter the given boundary and, if so, filtering the given boundary, including adjusting the filtering the given boundary depending at least in part on the cross boundary discontinuity measure, the first side discontinuity measure, and the second side discontinuity measure; and
storing the adaptively filtered, reconstructed frame in the frame buffer for use as the reference frame in subsequent motion compensation; and
an output buffer configured to store the encoded data, as part of a bitstream, for output.
2. The computer system of claim 1 , wherein the reconstructing the frame includes, for each of the multiple blocks of the frame:
reconstructing a prediction residual for the block using a variable-block-size inverse frequency transform;
computing a motion-compensated prediction for the block using one or more motion vectors to predict pixel values of the block; and
reconstructing the block based upon the prediction residual and the motion-compensated prediction.
3. The computer system of claim 2 , wherein the one or more boundaries that are adaptively filtered are boundaries between 8×8 blocks, even when transform size of the variable-block-size inverse frequency transform is smaller than 8×8.
4. The computer system of claim 1 , wherein the operations further comprise reconstructing a previous frame of the video sequence, including, for each of multiple blocks of the previous frame:
computing a spatial extrapolation for the block;
reconstructing a spatial-extrapolation residual for the block, including applying an inverse re-oriented frequency transform to the spatial-extrapolation residual; and
combining the spatial-extrapolation residual and the spatial extrapolation.
5. The computer system of claim 1 , wherein the operations further comprise selectively disabling the filtering for a subsequent frame of the video sequence.
6. The computer system of claim 1 , wherein the application of the filtering further depends at least in a part upon a quantization level.
7. The computer system of claim 1 , wherein the one or more boundaries are in a luminance plane of the reconstructed frame.
8. The computer system of claim 1 , wherein a single line of pixel values across the given boundary includes the multiple pixel values on the first side and the multiple pixel values on the second side.
9. One or more computer-readable memory or storage devices having stored thereon encoded data for a current frame of a video sequence, the encoded data comprising entropy-coded quantized transform coefficients and motion vectors for the current frame, wherein the encoded data is organized to facilitate decoding by operations that include:
reconstructing the current frame, including, for each of multiple blocks of the current frame:
computing a motion-compensated prediction for the block using one or more of the motion vectors to predict pixel values of the block;
reconstructing a prediction residual for the block using the entropy-coded quantized transform coefficients, including entropy decoding the entropy-coded quantized transform coefficients, inverse quantizing the decoded transform coefficients, and performing an inverse frequency transform using the inverse quantized transform coefficients; and
reconstructing the block based upon the prediction residual and the motion-compensated prediction;
adaptively filtering one or more boundaries between the multiple blocks in the reconstructed current frame to reduce boundary discontinuities, wherein application of the filtering across a given boundary of the one or more boundaries includes:
computing a cross boundary discontinuity measure that quantifies pixel value discontinuity across the given boundary, wherein the cross boundary discontinuity measure is computed using at least one pixel value on a first side of the given boundary and at least one pixel value on a second side of the given boundary, the first side of the given boundary being opposite the second side of the given boundary;
computing a first side discontinuity measure that quantifies pixel value discontinuity on the first side of the given boundary, wherein the first side discontinuity measure is computed using multiple pixel values on the first side of the given boundary;
computing a second side discontinuity measure that quantifies pixel value discontinuity on the second side of the given boundary, wherein the second side discontinuity measure is computed using multiple pixel values on the second side of the given boundary; and
determining whether to filter the given boundary and, if so, filtering the given boundary, including adjusting the filtering the given boundary depending at least in part on the cross boundary discontinuity measure, the first side discontinuity measure, and the second side discontinuity measure; and
storing the adaptively filtered, reconstructed current frame in a frame buffer for use as a reference frame in subsequent motion compensation.
10. The one or more computer-readable memory or storage devices of claim 9 , wherein the operations further comprise selectively disabling the filtering for a subsequent frame of the video sequence.
11. The one or more computer-readable memory or storage devices of claim 9 , wherein the application of the filtering further depends at least in a part upon a quantization level.
12. The one or more computer-readable memory or storage devices of claim 9 , wherein the one or more boundaries are in a luminance plane of the reconstructed current frame.
13. The one or more computer-readable memory or storage devices of claim 9 , wherein a single line of pixel values across the given boundary includes the multiple pixel values on the first side and the multiple pixel values on the second side.
14. In a computer system that implements a video decoder, a method comprising:
receiving encoded data, in a bitstream, for at least part of a video sequence; and
decoding the encoded data to reconstruct multiple frames of the video sequence, including:
reconstructing a current frame among the multiple frames of the video sequence;
adaptively filtering one or more boundaries between multiple blocks in the reconstructed current frame to reduce boundary discontinuities, wherein application of the filtering across a given boundary of the one or more boundaries includes:
computing a cross boundary discontinuity measure that quantifies pixel value discontinuity across the given boundary, wherein the cross boundary discontinuity measure is computed using at least one pixel value on a first side of the given boundary and at least one pixel value on a second side of the given boundary, the first side of the given boundary being opposite the second side of the given boundary;
computing a first side discontinuity measure that quantifies pixel value discontinuity on the first side of the given boundary, wherein the first side discontinuity measure is computed using multiple pixel values on the first side of the given boundary;
computing a second side discontinuity measure that quantifies pixel value discontinuity on the second side of the given boundary, wherein the second side discontinuity measure is computed using multiple pixel values on the second side of the given boundary; and
determining whether to filter the given boundary and, if so, filtering the given boundary, including adjusting the filtering the given boundary depending at least in part on the cross boundary discontinuity measure, the first side discontinuity measure, and the second side discontinuity measure; and
storing the adaptively filtered, reconstructed current frame for use as a reference frame in motion compensation for a subsequent frame among the multiple frames of the video sequence.
15. The method of claim 14 , wherein the reconstructing the current frame includes, for each of the multiple blocks of the current frame:
reconstructing a prediction residual for the block using a variable-block-size inverse frequency transform;
computing a motion-compensated prediction for the block using one or more motion vectors to predict pixel values of the block; and
reconstructing the block based upon the prediction residual and the motion-compensated prediction.
16. The method of claim 14 , wherein the decoding further comprises selectively disabling the filtering for another frame among the multiple frames of the video sequence.
17. The method of claim 14 , wherein the application of the filtering further depends at least in a part upon a quantization level.
18. The method of claim 14 , wherein the one or more boundaries that are adaptively filtered are boundaries between 8×8 blocks.
19. The method of claim 14 , wherein the one or more boundaries are in a luminance plane of the reconstructed current frame.
20. The method of claim 14 , wherein a single line of pixel values across the given boundary includes the multiple pixel values on the first side and the multiple pixel values on the second side.