Temporal and spatial video block reordering in a decoder to improve cache hits
This disclosure describes techniques in which the decoding order of video blocks is modified relative to the display order of video blocks. The decoding order may be modified temporally such that video blocks of different video frames (or other coded units) are decoded in an alternating manner. In this case, the decoding order of video blocks may alternate between video blocks of two or more different frames. Furthermore, the decoding order may also be modified spatially within a given video block such that the video blocks are decoded in an order that does not correspond to the raster scan order of the video blocks. The techniques may improve the use of memory by improving the likelihood of cache hits, thereby reducing the number of memory loads from an external memory to an internal cache associated with the decoder.
1. A method of decoding video data, the method comprising:
receiving a sequence of video frames in a decoder, wherein the sequence defines a display order of the video frames, each video frame including a plurality of video blocks;
identifying, at the decoder, a plurality of frames included in the sequence of video frames to be decoded in a decoding order different from the display order, wherein none of the plurality of frames comprises video blocks that are dependent on video blocks of another frame of the plurality of frames and wherein the plurality of frames includes at least a first video frame and a second video frame;
defining a statistical metric associated with the plurality of video blocks, wherein the statistical metric comprises at least one of an average motion across at least a portion of the sequence of video frames and an average motion across the first and second video frames;
loading, from a memory to a cache, predictive data for the plurality of frames based on the statistical metric, wherein the cache is in data communication with the decoder;
defining the decoding order of the video blocks of the identified plurality of frames based at least in part on the statistical metric, wherein the defined decoding order of the plurality of frames reduces a number of times predictive data is loaded from the memory to the cache, wherein the decoding order defines a spatial order and a temporal order in which the plurality of frames is decoded based on the statistical metric; and
decoding the video blocks of the plurality of frames using the predictive data in the cache according to the decoding order, wherein the decoding order identifies at least some video blocks included in the second video frame for decoding before at least some video blocks included in the first video frame, the blocks of the first video frame being further identified for decoding before completely decoding the second video frame, wherein the second video frame occurs after the first video frame in the display order.
2. The method of claim 1 , wherein the sequence of encoded video frames comprises at least one of:
a sequence of portions of video frames; and
a sequence of slices of video frames.
3. The method of claim 1 , wherein the decoding order alternates between one or more video blocks of the first encoded video frame and one or more video blocks of the second video frame.
4. The method of claim 1 , wherein the decoding order is based on a likelihood that the predictive data loaded in the cache is used in decoding the video blocks of the plurality of frames without needing to reload the cache.
5. The method of claim 1 , wherein the decoding order is defined such that the video blocks of the first video frame are decoded in an order that does not correspond to a raster scan order of the first video frame and the video blocks of the second video frame are decoded in an order that does not correspond to a raster scan order of the second video frame.
6. The method of claim 5 , wherein at least one video block of a second row of the first video frame is decoded prior to at least one video block of a first row of the first video frame, wherein the second row is below the first row.
7. The method of claim 1 , wherein the first video frame comprises a first B-frame and the second video frame comprises a second P-frame, and wherein the sequence of video frames follows an IBPBP format including an I-frame, followed by the first B-frame, followed by a first P-frame, followed by a second B-frame, followed by the second P-frame, wherein the identified plurality of frames consist of the first B-frame and the second P-frame.
8. The method of claim 1 , wherein the first video frame comprises a first B-frame and the second video frame comprises a second P-frame, and wherein the sequence of video frames follows an IBBPBBP format including an I-frame, followed by the first B-frame, followed by a second B-frame, followed by a first P-frame, followed by a third B-frame, followed by a fourth B-frame, followed by the second P-frame, wherein the identified plurality of frames consist of the first B-frame, the second B-frame, and the second P-frame.
9. The method of claim 1 , wherein the video blocks of the second video frame are decoded in parallel with the video blocks of the first video frame.
10. An apparatus that decodes video data, the apparatus comprising a video decoder and an associated cache, the video decoder being configured to:
receive a sequence of video frames wherein the sequence defines a display order of the video frames, each video frame including a plurality of video blocks;
identify a plurality of frames included in the sequence of video frames to be decoded in a decoding order different from the display order, wherein none of the plurality of frames comprises video blocks that are dependent on video blocks of another frame of the plurality of frames and wherein the plurality of frames includes at least a first video frame and a second video frame;
define a statistical metric associated with the plurality of video blocks, wherein the statistical metric comprises at least one of an average motion across at least a portion of the sequence of video frames and an average motion across the first and second video frames;
load, from a memory to the cache, predictive data for the plurality of frames based on the statistical metric;
define the decoding order of the video blocks of the identified plurality of frames based at least in part on the statistical metric, wherein the defined decoding order of the plurality of frames reduces a number of times predictive data is loaded from the memory to the cache, wherein the decoding order defines a spatial order and a temporal order in which the plurality of frames is decoded based on the statistical metric; and
decode the video blocks of the plurality of frames using the predictive data in the cache according to the decoding order, wherein the decoding order identifies at least some video blocks included in the second video frame for decoding before at least some video blocks included in the first video frame, the blocks of the first video frame being further identified for decoding before completely decoding the second video frame, wherein the second video frame occurs after the first video frame in the display order.
11. The apparatus of claim 10 , wherein the sequence of encoded video frames comprises at least one of:
a sequence of portions of video frames; and
a sequence of slices of video frames.
12. The apparatus of claim 10 , wherein the decoding order alternates between one or more video blocks of the first encoded video frame and one or more video blocks of the second video frame.
13. The apparatus of claim 10 , wherein the decoding order is based on a likelihood that the predictive data loaded in the cache is used in decoding the video blocks of the plurality of frames without needing to reload the cache.
14. The apparatus of claim 10 , wherein the decoding order is defined such that the video blocks of the first video frame are decoded in an order that does not correspond to a raster scan order of the first video frame and the video blocks of the second video frame are decoded in an order that does not correspond to a raster scan order of the second video frame.
15. The apparatus of claim 14 , wherein at least one video block of a second row of the first video frame is decoded prior to at least one video block of a first row of the first video frame, wherein the second row is below the first row.
16. The apparatus of claim 10 , wherein the first video frame comprises a first B-frame and the second video frame comprises a second P-frame, and wherein the sequence of video frames follow an IBPBP format including an I-frame, followed by the first B-frame, followed by a first P-frame, followed by a second B-frame, followed by the second P-frame, wherein the identified plurality of frames consist of the first B-frame and the second P-frame.
17. The apparatus of claim 10 , wherein the first video frame comprises a first B-frame and the second video frame comprises a second P-frame, and wherein the sequence of video frames follow an IBBPBBP format including an I-frame, followed by the first B-frame, followed by a second B-frame, followed by a first P-frame, followed by a third B-frame, followed by a fourth B-frame, followed by the second P-frame, wherein the identified plurality of frames consist of the first B-frame, the second B-frame and the second P-frame.
18. The apparatus of claim 10 , wherein the video decoder comprises an integrated circuit.
19. The apparatus of claim 10 , wherein the video decoder comprises a microprocessor.
20. The apparatus of claim 10 , wherein the apparatus comprises a wireless communication device that includes the video decoder.
21. A device that decodes video data, the device comprising:
means for receiving a sequence of video frames wherein the sequence defines a display order of the video frames, each video frame including a plurality of video blocks;
means for identifying a plurality of frames included in the sequence of video frames to be decoded in a decoding order different from the display order, wherein none of the plurality of frames comprises video blocks that are dependent on video blocks of another frame of the plurality of frames and wherein the plurality of frames includes at least a first video frame and a second video frame;
means for defining a statistical metric associated with the plurality of video blocks, wherein the statistical metric comprises at least one of an average motion across at least a portion of the sequence of video frames and an average motion across the first and second video frames;
means for loading, from a memory to a cache, predictive data for the plurality of frames based on the statistical metric;
means for defining the decoding order of the video blocks of the identified plurality of frames based at least in part on the statistical metric, wherein the defined decoding order of the plurality of frames reduces a number of times predictive data is loaded from the memory to the cache, wherein the decoding order defines a spatial order and a temporal order in which the plurality of frames is decoded based on the statistical metric; and
means for decoding the video blocks of the plurality of frames included in the video frames using the predictive data in the cache according to the decoding order, wherein the decoding order identifies at least some video blocks included in the second video frame for decoding before at least some video blocks included in the first video frame, the blocks of the first video frame being further identified for decoding before completely decoding the second video frame, wherein the second video frame occurs after the first video frame in the display order.
22. The device of claim 21 , wherein the sequence of encoded video frames comprises at least one of:
a sequence of portions of video frames; and
a sequence of slices of video frames.
23. The device of claim 21 , wherein the decoding order alternates between one or more video blocks of the first encoded video frame and one or more video blocks of the second video frame.
24. The device of claim 21 , wherein the decoding order is based on a likelihood that the predictive data loaded in the cache is used in decoding the video blocks of the plurality of frames without needing to reload the cache.
25. The device of claim 21 , wherein the decoding order is defined such that the video blocks of the first video frame are decoded in an order that does not correspond to a raster scan order of the first video frame and the video blocks of the second video frame are decoded in an order that does not correspond to a raster scan order of the second video frame.
26. The device of claim 25 , wherein at least one video block of a second row of the first video frame is decoded prior to at least one video block of a first row of the first video frame, wherein the second row is below the first row.
27. The device of claim 21 , wherein the first video frame comprises a first B-frame and the second video frame comprises a second P-frame, and wherein the sequence of video frames follow an IBPBP format including an I-frame, followed by the first B-frame, followed by a first P-frame, followed by a second B-frame, followed by the second P-frame, wherein the identified plurality of frames consist of the first B-frame and the second P-frame.
28. The device of claim 21 , wherein the first video frame comprises a first B-frame and the second video frame comprises a second P-frame, and wherein the sequence of video frames follow an IBBPBBP format including an I-frame, followed by the first B-frame, followed by a second B-frame, followed by a first P-frame, followed by a third B-frame, followed by a fourth B-frame, followed by the second P-frame, wherein the identified plurality of frames consist of the first B-frame, the second B-frame and the second P-frame.
29. A non-transitory computer-readable storage medium comprising instructions that upon execution in a processor cause the processor to:
upon receiving a sequence of video frames, wherein the sequence defines a display order of the video frames, each video frame including a plurality of video blocks;
identify a plurality of frames included in the sequence of video frames to be decoded in a decoding order different from the display order, wherein none of the plurality of frames comprises video blocks that are dependent on video blocks of another frame of the plurality of frames and wherein the plurality of frames includes at least a first video frame and a second video frame;
define a statistical metric associated with the plurality of video blocks, wherein the statistical metric comprises at least one of an average motion across at least a portion of the sequence of video frames and an average motion across the first and second video frames;
load, from a memory to a cache, predictive data for the plurality of frames based on the statistical metric;
define the decoding order of the video blocks of the identified plurality of frames based at least in part on the statistical metric, wherein the decoding order of the plurality of frames reduces a number of times predictive data is loaded from the memory to the cache, wherein the decoding order defines a spatial order and a temporal order in which the plurality of frames is decoded based on the statistical metric; and
decode the video blocks of the plurality of frames using the predictive data in the cache according to the decoding order, wherein the decoding order identifies at least some video blocks included in the second video frame for decoding before at least some video blocks included in the first video frame, the blocks of the first video frame being further identified for decoding before completely decoding the second video frame, wherein the second video frame occurs after the first video frame in the display order.
30. The computer-readable storage medium of claim 29 , wherein the sequence of encoded video frames comprises at least one of:
a sequence of portions of video frames; and
a sequence of slices of video frames.
31. The computer-readable storage medium of claim 29 , wherein the decoding order alternates between one or more video blocks of the first encoded video frame and one or more video blocks of the second video frame.
32. The computer-readable storage medium of claim 29 , wherein the decoding order is based on a likelihood that the predictive data loaded in the cache is used in decoding the video blocks of the plurality of frames without needing to reload the cache.
33. The computer-readable storage medium of claim 29 , wherein the decoding order is defined such that the video blocks of the first video frame are decoded in an order that does not correspond to a raster scan order of the first video frame and the video blocks of the second video frame are decoded in an order that does not correspond to a raster scan order of the second video frame.
34. The computer-readable storage medium of claim 33 , wherein at least one video block of a second row of the first video frame is decoded prior to at least one video block of a first row of the first video frame, wherein the second row is below the first row.
35. The computer-readable storage medium of claim 29 , wherein the first video frame comprises a first B-frame and the second video frame comprises a second P-frame, and wherein the sequence of video frames follow an IBPBP format including an I-frame, followed by the first B-frame, followed by a first P-frame, followed by a second B-frame, followed by the second P-frame, wherein the identified plurality of frames consist of the first B-frame and the second P-frame.
36. The computer-readable storage medium of claim 29 , wherein the first video frame comprises a first B-frame and the second video frame comprises a second P-frame, and wherein the sequence of video frames follow an IBBPBBP format including an I-frame, followed by the first B-frame, followed by a second B-frame, followed by a first P-frame, followed by a third B-frame, followed by a fourth B-frame, followed by the second P-frame, wherein the identified plurality of frames consist of the first B-frame, the second B-frame and the second P-frame.