Spatiotemporal prediction for bidirectionally predictive(B) pictures and motion vector prediction for multi-picture reference motion compensation
View Patent ↗Several improvements for use with Bidirectionally Predictive (B) pictures within a video sequence are provided. In certain improvements Direct Mode encoding and/or Motion Vector Prediction are enhanced using spatial prediction techniques. In other improvements Motion Vector prediction includes temporal distance and subblock information, for example, for more accurate prediction. Such improvements and other presented herein significantly improve the performance of any applicable video coding system/logic.
1. A method for use in encoding video data in a video encoder, the method comprising:
making a spatial/temporal motion vector prediction decision for at least one direct mode macroblock in a B-picture, wherein the spatial/temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock;
selectively encoding the at least one direct mode macroblock, wherein the encoding includes for a given direct mode macroblock of the at least one direct mode macroblock:
selecting a reference picture for the given direct mode macroblock from among reference pictures used for surrounding portions of the B-picture, wherein the selecting the reference picture for the given direct mode macroblock comprises selecting a minimum reference picture index for the given direct mode macroblock from among the reference picture indices used for the surrounding portions of the B-picture;
performing spatial motion vector prediction for the given direct mode macroblock; and
performing motion compensation for the given direct mode macroblock; and
signaling spatial/temporal motion vector prediction decision information for the at least one direct mode macroblock in a header that includes header information for plural macroblocks in the B-picture, wherein the signaling of the spatial/temporal motion vector prediction decision information in the header communicates to a video decoder the spatial/temporal motion vector prediction decision for the at least one direct mode macroblock.
2. The method of claim 1 wherein the plural macroblocks in the B-picture are in a slice of the B-picture.
3. The method of claim 1 wherein the at least one direct mode macroblock comprises plural direct mode macroblocks.
4. The method of claim 3 wherein the plural direct mode macroblocks are 16×16 macroblocks.
5. The method of claim 4 wherein each of the 16×16 macroblocks includes four 8×8 sub-blocks.
6. The method of claim 1 wherein the surrounding portions are surrounding macroblocks.
7. The method of claim 1 wherein the spatial motion vector prediction comprises median motion vector prediction.
8. The method of claim 1 further comprising:
analyzing the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the analysis.
9. The method of claim 1 further comprising:
analyzing motion flow within the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the analysis.
10. The method of claim 1 further comprising:
analyzing whether collocated blocks of a subsequent picture have zero motion, the subsequent picture following the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the analysis.
11. The method of claim 1 further comprising:
analyzing temporal distance between the B-picture and pictures around the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the analysis.
12. The method of claim 1 further comprising:
identifying a scene change around the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the identification of the scene change.
13. A method for use in decoding video data in a video decoder, the method comprising:
receiving signaled spatial/temporal motion vector prediction decision information for at least one direct mode macroblock in a header that includes header information for plural macroblocks in a B-picture;
from the signaled spatial/temporal motion vector prediction decision information in the header, determining a spatial/temporal motion vector prediction decision for the at least one direct mode macroblock, wherein the spatial or temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock; and
decoding the at least one direct mode macroblock, wherein the decoding includes for a given direct mode macroblock of the at least one direct mode macroblock:
selecting a reference picture for the given direct mode macroblock from among reference pictures used for surrounding portions of the B-picture, wherein the selecting the reference picture for the given direct mode macroblock comprises selecting a minimum reference picture index for the given direct mode macroblock from among the reference picture indices used for the surrounding portions of the B-picture;
performing spatial motion vector prediction for the given direct mode macroblock; and
performing motion compensation for the given direct mode macroblock.
14. The method of claim 13 wherein the plural macroblocks in the B-picture are in a slice of the B-picture.
15. The method of claim 13 wherein the at least one direct mode macroblock comprises plural direct mode macroblocks.
16. The method of claim 15 wherein the plural direct mode macroblocks are 16×16 macroblocks.
17. The method of claim 16 wherein each of the 16×16 macroblocks includes four 8×8 sub-blocks.
18. The method of claim 13 wherein the surrounding portions are surrounding macroblocks.
19. The method of claim 13 further comprising:
displaying visual results of the decoding of the video data; and
reproducing audio data associated with the video data.
20. A video decoder implemented with a computing device, wherein the video decoder is adapted to perform a method comprising:
receiving signaled spatial/temporal motion vector prediction decision information for at least one direct mode macroblock in a header that includes header information for plural macroblocks in a B-picture;
from the signaled spatial/temporal motion vector prediction decision information in the header, determining a spatial/temporal motion vector prediction decision for the at least one direct mode macroblock, wherein the spatial or temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock; and
decoding the at least one direct mode macroblock, wherein the decoding includes for a given direct mode macroblock of the at least one direct mode macroblock:
selecting a reference picture for the given direct mode macroblock from among reference pictures used for surrounding portions of the B-picture, wherein the selecting the reference picture for the given direct mode macroblock comprises selecting a minimum reference picture index for the given direct mode macroblock from among the reference picture indices used for the surrounding portions of the B-picture;
performing spatial motion vector prediction for the given direct mode macroblock; and
performing motion compensation for the given direct mode macroblock.
21. The video decoder of claim 20 wherein the plural macroblocks in the B-picture are in a slice of the B-picture.
22. The video decoder of claim 20 wherein the at least one direct mode macroblock comprises plural direct mode macroblocks.
23. The video decoder of claim 20 wherein the surrounding portions are surrounding macroblocks.
24. The video decoder of claim 20 wherein the computing device further includes an audio reproduction module for reproducing audio data.
25. The video decoder of claim 20 wherein the computing device is a hand-held device that includes a display, a network interface, one or more processors and memory.
26. The video decoder of claim 20 wherein the computing device is a portable communication device that includes a display, a network interface, one or more processors and memory.
27. The video decoder of claim 20 wherein the computing device is a set-top box that includes a network interface, one or more processors and memory.
28. One or more memory devices having stored thereon computer-executable instructions for causing a computing device programmed thereby to perform a method of decoding video data, the method comprising:
receiving signaled spatial/temporal motion vector prediction decision information for at least one direct mode macroblock in a header that includes header information for plural macroblocks in a B-picture;
from the signaled spatial/temporal motion vector prediction decision information in the header, determining a spatial/temporal motion vector prediction decision for the at least one direct mode macroblock, wherein the spatial or temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock; and
decoding the at least one direct mode macroblock, wherein the decoding includes for a given direct mode macroblock of the at least one direct mode macroblock:
selecting a reference picture for the given direct mode macroblock from among reference pictures used for surrounding portions of the B-picture, wherein the selecting the reference picture for the given direct mode macroblock comprises selecting a minimum reference picture index for the given direct mode macroblock from among the reference picture indices used for the surrounding portions of the B-picture;
performing spatial motion vector prediction for the given direct mode macroblock; and
performing motion compensation for the given direct mode macroblock.
29. The one or more memory devices of claim 28 wherein the surrounding portions are surrounding macroblocks.
30. The one or more memory devices of claim 28 wherein the surrounding portions are surrounding macroblocks.
31. A portable communication device that includes a display, a network interface, one or more processors, memory, an audio reproduction module for reproducing audio, and a video decoder, wherein the video decoder is adapted to perform a method comprising:
receiving signaled spatial/temporal motion vector prediction decision information for at least one direct mode macroblock in a header that includes header information for plural macroblocks in a slice of a B-picture;
from the signaled spatial/temporal motion vector prediction decision information in the header, determining a spatial/temporal motion vector prediction decision for the at least one direct mode macroblock, wherein the spatial or temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock; and
decoding the at least one direct mode macroblock, wherein the decoding includes for a given direct mode macroblock of the at least one direct mode macroblock:
selecting a reference picture for the given direct mode macroblock from among reference pictures used for surrounding portions of the B-picture, wherein the selecting the reference picture for the given direct mode macroblock comprises selecting a minimum reference picture index for the given direct mode macroblock from among the reference picture indices used for the surrounding portions of the B-picture;
performing spatial motion vector prediction for the given direct mode macroblock; and
performing motion compensation for the given direct mode macroblock.
32. A video encoder implemented with a computing device, wherein the video encoder is adapted to perform a method comprising:
analyzing one or more pictures of a video sequence;
based at least in part on results of the analyzing, making a spatial/temporal motion vector prediction decision for at least one direct mode macroblock in a B-picture among the one or more pictures of the video sequence, wherein the spatial/temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock;
selectively encoding the at least one direct mode macroblock, wherein the encoding includes for a given direct mode macroblock of the at least one direct mode macroblock:
selecting a reference picture for the given direct mode macroblock from among reference pictures used for surrounding portions of the B-picture, wherein the selecting the reference picture for the given direct mode macroblock comprises selecting a minimum reference picture index for the given direct mode macroblock from among the reference picture indices used for the surrounding portions of the B-picture;
performing spatial motion vector prediction for the given direct mode macroblock; and
performing motion compensation for the given direct mode macroblock; and
signaling spatial/temporal motion vector prediction decision information for the at least one direct mode macroblock in a header that includes header information for plural macroblocks in the B-picture, wherein the signaling of the spatial/temporal motion vector prediction decision information in the header communicates to a video decoder the spatial/temporal motion vector prediction decision for the at least one direct mode macroblock.
33. The video encoder of claim 32 wherein the plural macroblocks in the B-picture are in a slice of the B-picture.
34. The video encoder of claim 32 wherein the surrounding portions are surrounding macroblocks.
35. The video encoder of claim 32 wherein the analyzing comprises analyzing the B-picture.
36. The video encoder of claim 32 wherein the analyzing comprises analyzing motion flow within the B-picture.
37. The video encoder of claim 32 wherein the analyzing comprises analyzing whether collocated blocks of a subsequent picture have zero motion, the subsequent picture following the B-picture.
38. The video encoder of claim 32 wherein the analyzing comprises analyzing temporal distance between the B-picture and pictures around the B-picture.
39. The video encoder of claim 32 wherein the analyzing comprises identifying a scene change around the B-picture.