IP Library Granted Patent US 7,646,817
Granted Patent B2
US 7,646,817 · App. 10/402,882 · Granted Jan 12, 2010

Accelerating video decoding using a graphics processing unit

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,646,817
App. No.
10/402,882
Granted
Jan 12, 2010
Kind
B2
Abstract

An accelerated video decoding system utilizes a graphics processing unit to perform motion compensation, image reconstruction, and color space conversion processes, while utilizing a central processing unit to perform other decoding processes.

Claims (77)

1. A method comprising:

receiving an encoded video bitstream, the video bitstream comprising intra-coded frame data and inter-coded frame data, wherein the inter-coded frame data comprises multiple intra-coded macroblocks that do not depend on data from a corresponding reference macroblock and multiple inter-coded macroblocks that depend on data from a corresponding reference macroblock;

decoding the intra-coded frame data using a central processing unit;

transferring the intra-coded frame data to a graphics processing unit;

using the graphics processing unit to:

buffer a copy of the intra-coded frame data to be used as reference frame data for the inter-coded frame data; and

transmit the intra-coded frame data to a display device to be displayed;

decoding the inter-coded frame data using the central processing unit;

identifying two or more inter-coded macroblocks of the inter-coded frame data;

packing the two or more inter-coded macroblocks together into a data packet containing only inter-coded macroblocks;

transferring the data packet containing only inter-coded macroblocks to the graphics processing unit; and

using the graphics processing unit to:

apply a motion vector associated with the inter-coded frame data to the reference frame data to generate motion-compensated reference frame data;

reconstruct image data associated with the inter-coded frame data by adding the inter-coded frame data to the motion-compensated reference frame data; and

convert the intra-coded frame data from a first color space to a second color space, the second color space being a color space that is supported by the display device, and the first color space being a color space that is not supported by the display device.

2. The method as recited in claim 1 wherein the decoding the intra-coded frame data comprises performing a variable length decoding process.

3. The method as recited in claim 1 wherein the decoding the intra-coded frame data comprises performing a dequantization process.

4. The method as recited in claim 1 wherein the decoding the intra-coded frame data comprises applying an inverse discrete cosine transformation.

5. The method as recited in claim 1 , further comprising:

identifying two or more intra-coded macroblocks of the inter-coded frame data;

packing the two or more intra-coded macroblocks together into a data packet containing only intra-coded macroblocks; and

transferring the data packet containing only intra-coded macroblocks to the graphics processing unit.

6. A method comprising:

receiving an encoded video bitstream, the video bitstream comprising intra-coded frame data and inter-coded frame data, wherein the inter-coded frame data comprises multiple macroblocks wherein each macroblock has an associated motion compensation precision;

decoding the intra-coded frame data using a central processing unit;

ransferring the intra-coded frame data to a graphics processing unit;

sing the graphics processing unit to:

buffer a copy of the intra-coded frame data to be used as reference frame data for the inter-coded frame data; and

transmit the intra-coded frame data to a display device to be displayed; decoding the inter-coded frame data using the central processing unit;

identifying two or more macroblocks, each of the macroblocks having the same motion compensation precision;

packing the two or more macroblocks together into a data packet containing only macroblocks with the same motion compensation precision, wherein the motion compensation precision comprises motion compensation precision to the nearest half-pixel;

transferring the data packet to the graphics processing unit;

using the graphics processing unit to:

apply a motion vector associated with the inter-coded frame data to the reference frame data to generate motion-compensated reference frame data; and

reconstruct image data associated with the inter-coded frame data by adding the inter-coded frame data to the motion-compensated reference frame data.

7. The method as recited in claim 6 wherein identifying two or more macroblocks, each of the macroblocks having the same motion compensation precision comprises:

identifying two or more macroblocks, each of the macroblocks having an associated motion vector equal to zero.

8. The method as recited in claim 6 wherein the motion compensation precision is to the nearest pixel.

9. The method as recited in claim 6 wherein the motion compensation precision is to the nearest quarter-pixel.

10. The method as recited in claim 1 further comprising:

using the graphics processing unit to buffer a copy of the reconstructed image data to be used as reference data for subsequent inter-coded frame data.

11. The method as recited in claim 1 further comprising:

using the graphics processing unit to convert the reconstructed image data from a first color space to a second color space, the second color space being a color space that is supported by the display device.

12. A method comprising: receiving encoded video data; using a first processing unit to decompress the encoded video data; packing the two or more inter-coded macroblocks together into a data packet containing only inter-coded macroblocks; transferring the data packet containing only inter-coded macroblocks to a second processing unit; and using a second processing unit comprising a graphics processing unit to perform color space conversion processing in association with the video data to convert the intra-coded frame data from a first color space to a second color space, the second color space being a color space that is supported by the display device, and the first color space being a color space that is not supported by the display device.

13. The method as recited in claim 12 wherein the first processing unit is implemented as a central processing unit.

14. The method as recited in claim 12 wherein the first processing unit is implemented as a graphics processing unit.

15. The method as recited in claim 12 wherein the second processing unit is implemented as a graphics processing unit configured to perform vector-based graphics processing operations.

16. The method as recited in claim 12 wherein the second processing unit is implemented as a graphics processing unit configured to perform pixel-based graphics processing operations.

17. The method as recited in claim 12 further comprising:

using the second processing unit to perform predicted image reconstruction in association with the video data.

18. The method as recited in claim 12 further comprising:

using the second processing unit to perform motion compensation processing in association with the video data.

19. A system comprising: a central processing unit configured to perform a first operation in association with decoding a video bitstream; packing the two or more inter-coded macroblocks together into a data packet containing only inter-coded macroblocks; transferring the data packet containing only inter-coded macroblocks to a graphics processing unit; and the graphics processing unit comprising a programmable pixel shader component configured to perform color space conversion processing in association with decoding the video bitstream to convert the intra-coded frame data from a first color space to a second color space, the second color space being a color space that is supported by the display device, and the first color space being a color space that is not supported by the display device.

20. The system as recited in claim 19 , wherein the first operation comprises a video decompression operation.

21. The system as recited in claim 19 , wherein the first operation comprises at least one of inverse discrete cosine transformation, dequantization, or variable length decoding.

22. The system as recited in claim 19 , wherein the programmable pixel shader component is further configured to perform an image reconstruction operation in association with decoding the video bitstream.

23. The system as recited in claim 19 , wherein the graphics processing unit further comprises a programmable vertex shader component configured to perform a motion compensation operation in association with decoding the video bitstream.

24. The system as recited in claim 19 further comprising a buffer for storing decoded video bitstream data.

25. The system as recited in claim 19 further comprising a display device configured to display the video bitstream.

26. A system comprising: a central processing unit; a graphics processing unit; and a video decoding application configured to: direct the central processing unit to perform a first operation in association with decoding an encoded video bitstream; packing the two or more inter-coded macroblocks together into a data packet containing only inter-coded macroblocks; transferring the data packet containing only inter-coded macroblocks to a graphics processing unit; and direct the graphics processing unit to perform color space conversion processing in association with decoding the encoded video bitstream to convert the intra-coded frame data from a first color space to a second color space, the second color space being a color space that is supported by the display device, and the first color space being a color space that is not supported by the display device.

27. A computer-readable memory media having at least one tangible component, the computer-readable media encoded with computer executable instructions that, when executed, cause a computing system to: decompress a received video bitstream using a central processing unit; identify two or more macroblocks, each of the macroblocks having the same motion compensation precision; pack the two or more macroblocks together into a data packet containing only macroblocks with the same motion compensation precision, wherein the motion compensation precision comprises motion compensation precision to the nearest half-pixel; and perform motion compensation and color space conversion processing on the video bitstream using a graphics processing unit.

28. The computer-readable memory medium as recited in claim 27 , the computer-readable memory medium further encoded with computer executable instructions that, when executed, cause the computing system to:

in an event that a particular frame of the video bitstream is larger than a maximum data size that can be processed by the graphics processing unit, direct the central processing unit to partition data associated with the frame into overlapping sub-pictures, each of which are smaller than the maximum data size.

29. The computer-readable memory medium as recited in claim 27 , the computer-readable memory medium further encoded with computer executable instructions that, when executed, cause the computing system to use the graphics processing unit to apply a bi-linear filter to achieve half-pixel precision in association with the motion compensation processing.

30. The computer-readable memory medium as recited in claim 27 , the computer-readable memory medium further encoded with computer executable instructions that, when executed, cause the computing system to use the graphics processing unit to apply a bi-cubic filter to achieve quarter-pixel precision in association with the motion compensation processing.

31. The computer-readable memory medium as recited in claim 27 , the computer-readable memory medium further encoded with computer executable instructions that, when executed, cause the computing system to perform image reconstruction using the graphics processing unit.

32. The computer-readable memory medium as recited in claim 31 , the computer-readable media further encoded with computer executable instructions that, when executed, cause the computing system to:

reconstruct an image comprising Y, U, and V components such that the Y, U, and V components are arranged on a smallest target texture that is supported by the computer system and that is large enough to hold the Y, U, and V components of the image.

33. The method as recited in claim 6 further comprising:

using the graphics processing unit to convert the intra-coded frame data from a first color space to a second color space, the second color space being a color space that is supported by the display device.

34. The method as recited in claim 6 further comprising:

using the graphics processing unit to convert the reconstructed image data from a first color space to a second color space, the second color space being a color space that is supported by the display device.

35. The method as recited in claim 1 wherein the first color space comprises a YUV format and the second colors space comprises an RGB format.

36. The method as recited in claim 11 wherein the first color space comprises a YUV format and the second colors space comprises an RGB format.

37. The method as recited in claim 12 wherein the first color space comprises a YUV format and the second colors space comprises an RGB format.

38. The method as recited in claim 19 wherein the first color space comprises a YUV format and the second colors space comprises an RGB format.

39. The method as recited in claim 26 wherein the first color space comprises a YUV format and the second colors space comprises an RGB format.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034541/0477 →