IP Library Granted Patent US 8,861,591
Granted Patent B2
US 8,861,591 · App. 12/189,060 · Granted Oct 14, 2014

Software video encoder with GPU acceleration

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,861,591
App. No.
12/189,060
Granted
Oct 14, 2014
Kind
B2
Abstract

Embodiments of a software video encoder with GPU acceleration include a software encoder that partitions video processing tasks and assigns them to both a graphics processing unit (GPU) and a central processing unit (CPU). The partitioning and assigning is configurable for operation in different modes. The modes include a mode in which the total time for video processing (such as when transcoding a large existing file) is reduced, a mode in which less CPU cycles are consumed, thus freeing the CPU for other work, a mode in which the latency of processing (e.g., for video conferencing) is reduced, and a mode in which information from a game or other real-time activity being displayed on the screen is encoded.

Claims (95)

1. A data encoding method, comprising:

partitioning data processing tasks into subtasks among a plurality of processors, wherein:

the data processing tasks include processing video data;

at least one of the processors is a special purpose processor, and at least one of the processors is a central processing unit (CPU);

configuring the partitioning of the subtasks to the plurality of processors so as to perform the data processing in any one of a plurality of modes, including:

a mode in which a total time for data processing is reduced relative to other modes;

a mode in which fewer CPU cycles are consumed relative to other modes; or

a mode in which data processing latency is reduced relative to other modes;

choosing among tradeoffs available in each of the modes, including:

maintaining a predetermined minimum video quality and a predetermined maximum bitrate;

using the CPU to perform a coarse motion vector search;

performing an initial coarse motion vector search to establish a baseline set of macroblocks with motion predictions; and

computing the initial coarse motion vector search once to establish a baseline prediction, and computing the initial coarse motion vector search again utilizing the baseline prediction to minimize a differentially encoded motion vector; and

scheduling subtasks for each of the plurality of processors by a dispatcher, wherein the mode is used by the dispatcher to schedule the subtasks.

2. The method of claim 1 , wherein the plurality of modes further includes a mode in which a data stream from an application is encoded in real time.

3. The method of claim 2 , wherein the application is a video game being displayed on a display device concurrent with encoding of video data.

4. The method of claim 1 , wherein the total time for data processing includes a total time for video processing, including transcoding video data.

5. The method of claim 1 , further comprising performing the initial coarse motion vector search with different search ranges so as to minimize compute time.

6. The method of claim 2 , further comprising

intercepting a call to flip from a first buffer to a second buffer;

placing a copy of a current frame stored in the first buffer in a queue for encoding; and

calling a previous address of the intercepted call such that previous processing continues.

7. The method of claim 1 , further comprising:

receiving video input frames;

performing motion estimation on the video received frames, including:

performing a hierarchical motion search to find motion vectors with an optimum sum of absolute difference (SAD) values, wherein SAD comprises a sum of absolute transformed difference (SATD); and

performing spatial filtering of the motion vectors, wherein spatial filtering includes making two or more motion vectors equal to achieve a zero differential.

8. A video data capture system, comprising:

at least one video data source including a central processing unit (CPU) running a video application;

at least one graphics processing unit (GPU) coupled to the video data source for receiving video frames, the at least one GPU configurable to perform a data encoding method, the method comprising,

partitioning data processing tasks into subtasks among a plurality of processors, wherein:

the data processing tasks include processing video data;

at least one of the processors is a special purpose processor, and at least one of the processors is a central processing unit (CPU);

configuring the partitioning of the subtasks to the plurality of processors so as to perform the data processing in any one of a plurality of modes, including:

a mode in which a total time for data processing is reduced relative to other modes;

a mode in which fewer CPU cycles are consumed relative to other modes; or

a mode in which data processing latency is reduced relative to other modes;

choosing among tradeoffs available in each of the modes, including:

maintaining a predetermined minimum video quality and a predetermined maximum bitrate;

using the CPU to perform a coarse motion vector search;

performing an initial coarse motion vector search to establish a baseline set of macroblocks with motion predictions; and

computing the initial coarse motion vector search once to establish a baseline prediction, and computing the initial coarse motion vector search again utilizing the baseline prediction to minimize a differentially encoded motion vector; and

scheduling subtasks for each of the plurality of processors by a dispatcher, wherein the mode is used by the dispatcher to schedule the subtasks.

9. A non-transitory computer readable medium having instructions stored thereon, which when executed cause a data encoding method to be performed, the method comprising:

partitioning data processing tasks into subtasks among a plurality of processors, wherein:

the data processing tasks include processing video data;

at least one of the processors is a special purpose processor, and at least one of the processors is a central processing unit (CPU);

configuring the partitioning of the subtasks to the plurality of processors so as to perform the data processing in any one of a plurality of modes, including:

a mode in which a total time for data processing is reduced relative to other modes;

a mode in which fewer CPU cycles are consumed relative to other modes; or

a mode in which data processing latency is reduced relative to other modes;

choosing among tradeoffs available in each of the modes, including:

maintaining a predetermined minimum video quality and a predetermined maximum bitrate;

using the CPU to perform a coarse motion vector search;

performing an initial coarse motion vector search to establish a baseline set of macroblocks with motion predictions; and

computing the initial coarse motion vector search once to establish a baseline prediction, and computing the initial coarse motion vector search again utilizing the baseline prediction to minimize a differentially encoded motion vector; and

scheduling subtasks for each of the plurality of processors by a dispatcher, wherein the mode is used by the dispatcher to schedule the subtasks.

10. The non-transitory computer readable medium of claim 9 , wherein the plurality of modes further includes a mode in which a data stream from an application is encoded in real time.

11. The non-transitory computer readable medium of claim 10 wherein the application is a video game being displayed on a display device concurrent with encoding of video data.

12. The non-transitory computer readable medium of claim 9 , wherein the total time for data processing includes a total time for video processing, including transcoding video data.

13. The non-transitory computer readable medium of claim 9 , wherein the method further comprises performing the initial coarse motion vector search with different search ranges so as to minimize compute time.

14. The non-transitory computer readable medium of claim 10 , wherein the method further comprises:

intercepting a call to flip from a first buffer to a second buffer;

placing a copy of a current frame stored in the first buffer in a queue for encoding; and

calling a previous address of the intercepted call such that previous processing continues.

15. The non-transitory computer readable medium of claim 9 , wherein the method further comprises:

receiving video input frames;

performing motion estimation on the video received frames, including:

performing a hierarchical motion search to find motion vectors with an optimum sum of absolute difference (SAD) values, wherein SAD comprises a sum of absolute transformed difference (SATD); and

performing spatial filtering of the motion vectors, wherein spatial filtering includes making some pairs of motion vectors the same to achieve a zero differential.

16. A system, comprising:

at least one central processing unit (CPU);

at least one special-purpose processor coupled to the at least one CPU;

wherein:

the at least one special-purpose processor is configurable to:

share a data encoding task with the at least one CPU, and

stream encoded data to a network; and

the data encoding task includes processing video data:

a dispatcher coupled to the at least one CPU and the at least one special-purpose processor, the dispatcher configurable to:

partition the data encoding task into subtasks and schedule the subtasks between the at least one CPU and the at least one special-purpose processor in any one of a plurality of modes, including:

a mode in which a total time for data processing is reduced relative to other modes;

a mode in which fewer CPU cycles are consumed relative to other modes; or

a mode in which data processing latency is reduced relative to other modes;

choose among tradeoffs available in each of the modes, including:

maintain a predetermined minimum video quality and a predetermined maximum bitrate;

use the at least one CPU to perform a coarse motion vector search:

perform an initial coarse motion vector search to establish a baseline set of macroblocks with motion predictions; and

compute the initial coarse motion vector search once to establish a baseline prediction, and compute the initial coarse motion vector search again utilizing the baseline prediction to minimize a differentially encoded motion vector; and

wherein the mode is used by the dispatcher to schedule the subtasks;

whereby the at least one CPU and the at least one special-purpose processor share the data encoding task; and

at least one receiving device coupled to the at least one CPU and to the at least one special-purpose processor via the network, wherein the at least one receiving device is configurable to receive the streamed encoded data from the network in real-time.

17. The system of claim 16 , wherein the at least one receiving device is further configurable to decode the encoded data.

18. The system of claim 16 , wherein the at least one special-purpose processor is a graphics processing unit (GPU), and wherein the data encoding task is video data encoding.

19. The system of claim 16 , wherein the network is the Internet.

20. The system of claim 18 , wherein a GPU and a CPU are on a same integrated circuit package.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2025
From: ONESTA IP LLC
To: MAGNUSON RESEARCH LLC
Reel/Frame 071352/0575 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2024
From: ADVANCED MICRO DEVICES, INC.
To: ONESTA IP, LLC
Reel/Frame 069381/0951 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2008
From: SCHMIT, MICHAEL L.; RAWTHER, RAJY MEEYAKHAN; GIDUTHURI, RADHA
To: ADVANCED MICRO DEVICES, INC.
Reel/Frame 021595/0300 →