Memory word array organization and prediction combination for memory access
View Patent ↗Described systems and methods allow a reduction in the memory bandwidth required in video coding (decoding/encoding) applications. According to a first aspect, the data assigned to each memory word is chosen to correspond to a 2D subarray of a larger array such as a macroblock. An array memory word organization allows reducing both the average and worst-case bandwidth required to retrieve predictions from memory in video coding applications, particularly for memory word sizes (memory bus widths) larger than the size of typical predictions. According to a second aspect, two or more 2D subarrays such as video predictions are retrieved from memory simultaneously as part of a larger 2D array, if retrieving the larger array requires fewer clock cycles than retrieving the subarrays individually. Allowing the combination of multiple predictions in one memory access operation can lead to a reduction in the average bandwidth required to retrieve predictions from memory.
1. A video coding method comprising:
determining a first number of memory access clock cycles needed to retrieve a first prediction according to motion compensation from random access memory;
determining a second number of memory access clock cycles needed to retrieve a second prediction according to motion compensation from the random access memory, the second prediction being distinct from the first prediction;
determining a third number of memory access clock cycles needed to retrieve a combined prediction containing the first prediction and the second prediction from the random access memory;
comparing the third number of clock cycles with a sum of the first number of clock cycles and the second number of clock cycles; and
sending to the random access memory a memory read command to retrieve the combined prediction from the random access memory when the third number of clock cycles is less than or equal to the sum of the first number of clock cycles and the second number of clock cycles.
2. The method of claim 1 , further comprising adding the first prediction to a first residual to generate a first target data set, and adding the second prediction to a second residual to generate a second target data set.
3. The method of claim 1 , further comprising performing a first subtraction operation between a first target data set and the first prediction to generate a first residual, and a second subtraction operation between a second target data set and the second prediction to generate a second residual.
4. The method of claim 1 , wherein determining the third number of memory access clock cycles comprises determining that the first prediction overlaps the second prediction.
5. The method of claim 1 wherein determining the third number of memory access clock cycles comprises determining that the first prediction does not overlap the second prediction.
6. The method of claim 1 , wherein determining the third number of memory access clock cycles comprises determining that an area of the combined prediction is defined by a rectangle of minimal area that contains an area of the first prediction and an area of the second prediction and that contains an integer number of memory words.
7. The method of claim 1 , further comprising evaluating whether to combine the first prediction with each of a plurality of predictions.
8. The method of claim 7 , further comprising evaluating whether to combine the combined prediction with each of at least a subset of the plurality of predictions.
9. The method of claim 1 , further comprising retrieving the combined prediction from the random access memory when the third number of clock cycles is less than or equal to the sum of the first number of clock cycles and the second number of clock cycles.
10. The method of claim 1 , wherein determining the third number of memory access clock cycles comprises determining that the combined prediction includes an image part present in neither of the first prediction nor of the second prediction.
11. The method of claim 1 , wherein in determining the first number of memory access clock cycles and determining the second number of memory access clock cycles, each of the first prediction and the second prediction is smaller than or equal to a macroblock.
12. The method of claim 11 , wherein in determining the first number of memory access clock cycles and determining the second number of memory access clock cycles, each of the first prediction and the second prediction has a luminance size of 4×4, 4×8, 8×4, 8×8, 16×8 or 16×16.
13. A video coding method comprising:
comparing a memory bandwidth requirement for retrieving a combined prediction containing a first prediction according to motion compensation and a second prediction according to motion compensation from a random access memory with a memory bandwidth requirement for retrieving individually the first prediction and the second prediction from the random access memory; and
retrieving from the random access memory either the combined prediction or the first and second predictions individually according to a comparison of the memory bandwidth requirement for retrieving the combined prediction to the memory bandwidth requirement for retrieving individually the first prediction and the second prediction.
14. The method of claim 13 , wherein in comparing the memory bandwidth requirement for retrieving a combined prediction, the first prediction overlaps the second prediction.
15. The method of claim 13 , wherein in comparing the memory bandwidth requirement for retrieving a combined prediction, the first prediction does not overlap the second prediction.
16. The method of claim 13 , wherein in comparing the memory bandwidth requirement for retrieving a combined prediction, an area of the combined prediction is defined by a rectangle of minimal area that contains an area of the first prediction and an area of the second prediction, and that contains an integer number of memory words.
17. The method of claim 13 , further comprising evaluating whether to combine the first prediction with each of a plurality of predictions.
18. The method of claim 17 , further comprising evaluating whether to combine the combined prediction with each of at least a subset of the plurality of predictions.
19. The method of claim 13 , comprising retrieving the combined prediction when the memory bandwidth requirement for retrieving the combined prediction is not larger than the memory bandwidth requirement for retrieving individually the first prediction.
20. The method of claim 13 , wherein in comparing the memory bandwidth requirement for retrieving a combined prediction, the combined prediction includes an image part present in neither of the first prediction nor of the second prediction.
21. The method of claim 13 , wherein in comparing the memory bandwidth requirement for retrieving a combined prediction, each of the first prediction and the second prediction is smaller than or equal to a macroblock.
22. The method of claim 21 , wherein in comparing the memory bandwidth requirement for retrieving a combined prediction, each of the first prediction and the second prediction has a luminance size of 4×4, 4×8, 8×4, 8×8, 16×8 or 16×16.
23. A video coding apparatus comprising:
comparison logic for comparing a memory bandwidth requirement for retrieving a combined prediction containing a first prediction according to motion compensation and a second prediction according to motion compensation from a random access memory with a memory bandwidth requirement for retrieving individually the first prediction and the second prediction from the random access memory; and
memory access logic connected to the comparison logic, for retrieving the combined prediction or the first and second predictions individually according to a result of complaining the memory bandwidth requirement for retrieving the combined prediction to the memory bandwidth requirement for retrieving individually the first prediction and the second prediction.
24. The apparatus of claim 23 , wherein the comparison logic further determines that the first prediction overlaps the second prediction.
25. The apparatus of claim 23 , wherein the memory access logic is configured to retrieve the combined prediction when the memory bandwidth requirement for retrieving the combined prediction is not larger than the memory bandwidth requirement for retrieving individually the first prediction and the second prediction.
26. The apparatus of claim 23 , wherein the comparison logic further determines that the combined prediction includes an image part present in neither of the first prediction and the second prediction.
27. The apparatus of claim 23 , wherein the comparison logic further receives the first prediction and the second prediction, and each of the first prediction and the second prediction is smaller than or equal to a macroblock.
28. The apparatus of claim 27 , wherein the comparison logic further receives the first prediction and the second prediction, and each of the first prediction and the second prediction has a luminance size of 4×4, 4×8, 8×4, 8×8, 16×8 or 16×16.
29. A video coding apparatus comprising:
means for comparing a memory bandwidth requirement for retrieving a combined prediction containing a first prediction according to motion compensation and a second prediction according to motion compensation from a random access memory with a memory bandwidth requirement for retrieving individually the first prediction and the second prediction from the random access memory; and
means for retrieving the combined prediction or the first and second predictions individually according to a result of comparing of the memory bandwidth requirement for retrieving the combined prediction to the memory bandwidth requirement for retrieving individually the first prediction and the second prediction.
30. The apparatus of claim 29 , wherein the means for comparing determine that the first prediction overlaps the second prediction.
31. The apparatus of claim 29 , wherein the means for retrieving either the combined prediction or the first and second predictions individually are configured to retrieve the combined prediction when the memory bandwidth requirement for retrieving the combined prediction is not larger than the memory bandwidth requirement for retrieving individually the first prediction and the second prediction.
32. The apparatus of claim 29 , wherein the means for comparing determine that the combined prediction includes an image part present in neither of the first prediction and the second prediction.
33. The apparatus of claim 29 , wherein the means for comparing receive the first prediction and the second prediction and each of the first prediction and the second prediction is smaller than or equal to a macroblock.
34. The apparatus of claim 33 , wherein the means for comparing receive the first prediction and the second prediction and each of the first prediction and the second prediction has a luminance size of 4×4, 4×8, 8×4, 8×8, 16×8 or 16×16.