Video coding / decoding with motion resolution switching and sub-block transform sizes
View Patent ↗Techniques and tools for video coding/decoding with motion resolution switching and sub-block transform coding/decoding are described. For example, a video encoder adaptively switches the resolution of motion estimation and compensation between quarter-pixel and half-pixel resolutions; a corresponding video decoder adaptively switches the resolution of motion compensation between quarter-pixel and half-pixel resolutions. For sub-block transform sizes, for example, a video encoder adaptively switches between 8×8, 8×4, and 4×8 DCTs when encoding 8×8 prediction residual blocks; a corresponding video decoder switches between 8×8, 8×4, and 4×8 inverse DCTs during decoding.
1. In a computer system that implements a video decoder, a method comprising:
receiving, in a bitstream, encoded video data for a video frame of a video sequence; and
with the computer system that implement the video decoder, decoding the video frame, including:
selecting a fractional pixel motion resolution, indicating motion vector resolution, for the video frame from among plural different fractional pixel motion resolutions,
wherein each of the plural different fractional pixel motion resolutions is less than single integer-pixel motion resolution,
the plural different fractional pixel motion resolutions including a first fractional pixel motion resolution and
a second fractional pixel motion resolution higher than the first fractional pixel motion resolution;
computing a motion-compensated prediction for a current block in the video frame using one or more motion vectors to predict pixel values of the current block, including:
using the second fractional pixel motion resolution for the one or more motion vectors for the current block
(1) if the selected fractional pixel motion resolution for the frame is the second fractional pixel motion resolution, and
(2) depending on value of the one or more motion vectors for the current block; and
otherwise, using the first fractional pixel motion resolution for the one or more motion vectors for the current block;
reconstructing a motion-compensated prediction residual for the current block in the video frame; and
reconstructing the current block in the video frame based upon the motion-compensated prediction residual and the motion-compensated prediction.
2. The method of claim 1 wherein the reconstructing the motion-compensated prediction residual uses a variable-block-size inverse frequency transform with support for switching of transform size at varying levels within the video frame.
3. The method of claim 2 wherein the bitstream includes switching information that indicates the transform size used for the motion-compensated prediction residual, and wherein format of the bitstream allows signaling of the switching information within the bitstream at frame level for the video frame or at macroblock level.
4. The method of claim 1 further comprising, for another block of pixel values:
computing a spatial extrapolation for the other block;
reconstructing a spatial-extrapolation residual for the other block, including applying an inverse re-oriented frequency transform to the spatial-extrapolation residual; and
combining the spatial-extrapolation residual and the spatial extrapolation.
5. The method of claim 1 wherein the selecting the fractional pixel motion resolution occurs on a frame-by-frame basis, and wherein the bitstream includes information indicating the selected fractional pixel motion resolution for the video frame.
6. The method of claim 1 wherein the plural different fractional pixel motion resolutions include quarter-pixel motion resolution.
7. The method of claim 1 further comprising:
selecting a filter for sub-pixel interpolation from among multiple filters, wherein the multiple filters include:
a first filter that tends to smooth pixel values; and
a second filter that tends to preserve edges; and
performing the sub-pixel interpolation using the selected filter.
8. The method of claim 1 wherein the value of the one or more motion vectors for the current block is a predicted value, and wherein the bitstream includes additional motion vector information to be combined with the predicted value.
9. A computer-readable medium storing computer-executable instructions for causing a processor, when programmed thereby, to perform:
receiving, in a bitstream, encoded video data for a video frame of a video sequence; and
decoding the video frame, including:
selecting a fractional pixel motion resolution, indicating motion vector resolution, for the video frame from among plural different fractional pixel motion resolutions,
wherein each of the plural different fractional pixel motion resolutions is less than single integer-pixel motion resolution,
the plural different fractional pixel motion resolutions including a first fractional pixel motion resolution and
a second fractional pixel motion resolution higher than the first fractional pixel motion resolution;
computing a motion-compensated prediction for a current block in the video frame using one or more motion vectors to predict pixel values of the current block, including:
using the second fractional pixel motion resolution for the one or more motion vectors for the current block
(1) if the selected fractional pixel motion resolution for the frame is the second fractional pixel motion resolution, and
(2) depending on a value of the one or more motion vectors for the current block; and
otherwise, using the first fractional pixel motion resolution for the one or more motion vectors for the current block;
reconstructing a motion-compensated prediction residual for the current block in the video frame; and
reconstructing the current block in the video frame based upon the motion-compensated prediction residual and the motion-compensated prediction.
10. The computer-readable medium of claim 9 wherein the reconstructing the motion-compensated prediction residual uses a variable-block-size inverse frequency transform with support for switching of transform size at varying levels within the video frame.
11. The computer-readable medium of claim 10 wherein the bitstream includes switching information that indicates the transform size used for the motion-compensated prediction residual, and wherein format of the bitstream allows signaling of the switching information within the bitstream at frame level for the video frame or at macroblock level.
12. The computer-readable medium of claim 9 further storing computer-executable instructions for causing the processor, when programmed thereby, to perform, for another block of pixel values:
computing a spatial extrapolation for the other block;
reconstructing a spatial-extrapolation residual for the other block, including applying an inverse re-oriented frequency transform to the spatial-extrapolation residual; and
combining the spatial-extrapolation residual and the spatial extrapolation.
13. The computer-readable medium of claim 9 wherein the selecting the fractional pixel motion resolution occurs on a frame-by-frame basis, and wherein the bitstream includes information indicating the selected fractional pixel motion resolution for the video frame.
14. The computer-readable medium of claim 9 wherein the plural different fractional pixel motion resolutions include quarter-pixel motion resolution.
15. The computer-readable medium of claim 9 further storing computer-executable instructions for causing the processor, when programmed thereby, to perform:
selecting a filter for sub-pixel interpolation from among multiple filters, wherein the multiple filters include:
a first filter that tends to smooth pixel values; and
a second filter that tends to preserve edges; and
performing the sub-pixel interpolation using the selected filter.
16. The computer-readable medium of claim 9 wherein the value of the one or more motion vectors for the current block is a predicted value, and wherein the bitstream includes additional motion vector information to be combined with the predicted value.
17. A video decoding system comprising a processor and memory, wherein the video decoding system is configured to perform:
receiving, in a bitstream, encoded video data for a video frame of a video sequence; and
decoding the video frame, including:
selecting a fractional pixel motion resolution, indicating motion vector resolution, for the video frame from among plural different fractional pixel motion resolutions,
wherein each of the plural different fractional pixel motion resolutions is less than single integer-pixel motion resolution,
the plural different fractional pixel motion resolutions including a first fractional pixel motion resolution and
a second fractional pixel motion resolution higher than the first fractional pixel motion resolution;
computing a motion-compensated prediction for a current block in the video frame using one or more motion vectors to predict pixel values of the current block, including:
using the second fractional pixel motion resolution for the one or more motion vectors for the current block
(1) if the selected fractional pixel motion resolution for the frame is the second fractional pixel motion resolution, and
(2) depending on a value of the one or more motion vectors for the current block; and
otherwise, using the first fractional pixel motion resolution for the one or more motion vectors for the current block;
reconstructing a motion-compensated prediction residual for the current block in the video frame; and
reconstructing the current block in the video frame based upon the motion-compensated prediction residual and the motion-compensated prediction.
18. The video decoding system of claim 17 wherein the selecting the fractional pixel motion resolution occurs on a frame-by-frame basis, and wherein the bitstream includes information indicating the selected fractional pixel motion resolution for the video frame.
19. The video decoding system of claim 17 wherein the plural different fractional pixel motion resolutions include quarter-pixel motion resolution.
20. The video decoding system of claim 17 wherein the value of the one or more motion vectors for the current block is a predicted value, and wherein the bitstream includes additional motion vector information to be combined with the predicted value.