Submission and synchronization techniques for scheduling and load balancing hardware accelerated tasks on heterogeneous platforms
Techniques related to scheduling and load balancing media tasks across heterogeneous hardware units are discussed. Such techniques include estimating completion times of received media tasks and, after submission of the media task to a selected hardware, only checking status of the media task after the estimated completion time.
1. A system to schedule media tasks, the system comprising:
a memory to store a media task;
machine-readable instructions; and
at least one programmable circuit to be programmed based on the machine-readable instructions to:
determine estimated durations to respectively process the media task at corresponding ones of a plurality of heterogeneous hardware units;
submit the media task to a selected one of the heterogeneous hardware units based on the estimated durations;
determine an estimated completion time for the media task based on addition of the estimated duration corresponding to the selected one of the heterogeneous hardware units to a start time corresponding to submission of the media task;
assign the media task to a thread, the thread to manage the media task; and
obtain, from the thread, a status of the media task at the selected one of the heterogeneous hardware units after, but not before, the estimated completion time has passed.
2. The system of claim 1 , wherein the media task is a first media task, the estimated completion time is a first estimated completion time, and one or more of the at least one programmable circuit is to:
determine the first estimated completion time for the first media task is within a threshold of a second estimated completion time for a second media task; and
create the thread based on the first estimated completion time being within the threshold of the second estimated completion time.
3. The system of claim 1 , wherein the media task is a first media task, the estimated completion time is a first estimated completion time, and one or more of the at least one programmable circuit to:
determine the first estimated completion time for the first media task corresponds to a second estimated completion time for a second media task; and
create the thread based on the first estimated completion time corresponding to the second estimated completion time.
4. The system of claim 2 , wherein the media task is a first media task, the estimated completion time is a first estimated completion time, the thread is a first thread, the second media task is assigned to a second thread, and one or more of the at least one programmable circuit is further to:
determine a third estimated completion time for a third media task is after the first estimated completion time, and determine a fourth estimated completion time for a fourth media task is after the third estimated completion time, the third media task subsequent to the first media task in a task queue and the fourth media task subsequent to the third media task in the task queue;
assign the third media task to the second thread based on the third media task being subsequent to the first media task in the task queue and the third estimated completion time being after the first estimated completion time and outside the threshold of the first estimated completion time; and
assign the fourth media task to the first thread based on the fourth media task being subsequent to the third media task in the task queue and the fourth estimated completion time being after the third estimated completion time and outside the threshold of the third estimated completion time.
5. The system of claim 1 , wherein the media task is a first media task, and one or more of the at least one programmable circuit is to:
obtain an indication that a latency time measure corresponding to completion of one or more second media tasks submitted to one or more of the heterogeneous hardware units prior to the submission of the first media task exceeds a threshold; and
create the thread based on the latency time measure exceeding the threshold.
6. The system of claim 5 , wherein the estimated completion time is a first estimated completion time, and the latency time measure is based on a difference between a time of report of output media data to an application and a second estimated completion time for at least one of the one or more second media tasks.
7. The system of claim 1 , wherein one or more of the at least one programmable circuit is to determine the estimated durations based on a task description corresponding to the media task and based on, for ones of the plurality of heterogeneous hardware units, respective a priori reference data corresponding to the task description.
8. The system of claim 1 , wherein the media task is a first media task, and one or more of the at least one programmable circuit is to:
determine a previous execution duration to process a second media task having a same type as the first media task at the selected one of the heterogeneous hardware units; and
set the estimated duration for the selected one of the heterogeneous hardware units based at least in part on the previous execution duration.
9. The system of claim 1 , wherein one or more of the at least one programmable circuit is to:
perform look ahead processing on the media task to generate one or more look ahead processing parameters for the media task, the look ahead processing to perform a same task as the media task with one or more differing processing characteristics; and
set the estimated duration based at least in part on the look ahead processing parameters.
10. The system of claim 1 , wherein one or more of the at least one programmable circuit is to:
after the status is obtained from the thread, obtain an indication that the media task is incomplete; and
provide, after the indication that the media task is incomplete, at least one of a polling inquiry or a blocking call command to the selected one of the heterogeneous hardware units.
11. The system of claim 1 , wherein one or more of the at least one programmable circuit is to select the selected one of the heterogeneous hardware units based on the selected one of the heterogeneous hardware units having a minimum estimated duration of the estimated durations.
12. The system of claim 1 , wherein the media task includes at least one of a frame encode, a frame decode, a motion vector search, a color conversion, a denoise, a color adjustment, or a frame resizing, and the heterogeneous hardware units include at least one of a graphics processing unit, a fixed function hardware block, an execution unit, or a system on a chip.
13. A method to schedule media tasks, the method comprising:
determining estimated durations to respectively process a media task at corresponding ones of a plurality of heterogeneous hardware units;
submitting the media task to a selected one of the heterogeneous hardware units based on the estimated durations;
determining an estimated completion time for the media task based on adding the estimated duration corresponding to the selected one of the heterogeneous hardware units to a start time corresponding to submission of the media task;
assigning the media task to a thread, the thread to manage the media task; and
obtaining, from the thread, a status of the media task at the selected one of the heterogeneous hardware units after, but not before, the estimated completion time.
14. The method of claim 13 , wherein the media task is a first media task, the estimated completion time is a first estimated completion time, and the assigning of the first media task to the thread includes:
determining the first estimated completion time for the first media task corresponds to a second estimated completion time for a second media task; and
creating the thread based on the first estimated completion time being within a threshold of the second estimated completion time.
15. The method of claim 13 , wherein the media task is a first media task, and the assigning of the media task to the thread includes:
obtaining an indication that a latency time measure corresponding to completion of one or more second media tasks submitted to one or more of the heterogeneous hardware units prior to the submission of the first media task exceeds a threshold; and
creating the thread based on the latency time measure exceeding the threshold.
16. The method of claim 13 , wherein the media task is a first media task, and the determining of the estimated durations includes:
determining a previous execution duration to process a second media task having a same type as the first media task at the selected one of the heterogeneous hardware units; and
setting the estimated duration for the selected one of the heterogeneous hardware units based at least in part on the previous execution duration.
17. The method of claim 13 , wherein the determining of the estimated durations includes:
performing look ahead processing on the media task to generate one or more look ahead processing parameters for the media task, the look ahead processing performing a same task as the media task with one or more differing processing characteristics; and
setting the estimated duration based at least in part on the look ahead processing parameters.
18. At least one memory comprising instructions to cause at least one programmable circuit to at least:
determine estimated durations to respectively process a media task at corresponding ones of a plurality of heterogeneous hardware units;
submit the media task a selected one of the heterogeneous hardware units based on the estimated durations;
determine an estimated completion time for the media task based on addition of the estimated duration corresponding to the selected one of the heterogeneous hardware units to a start time corresponding to submission of the media task;
assign the media task to a thread, the thread to manage the media task; and
obtain, from the thread, a status of the media task at the selected one of the heterogeneous hardware units after, but not before, the estimated completion time has passed.
19. The at least one memory of claim 18 , wherein the media task is a first media task, the estimated completion time is a first estimated completion time, and the instructions are to cause one or more of the at least one programmable circuit to:
determine the first estimated completion time for the first media task corresponds to a second estimated completion time for a second media task; and
create the thread based on the first estimated completion time being within a threshold of the second estimated completion time.
20. The at least one memory of claim 18 , wherein the media task is a first media task, the estimated completion time is a first estimated completion time, and the instructions are to cause one or more of the at least one programmable circuit to:
obtain an indication that a latency time measure corresponding to completion of one or more second media tasks submitted to one or more of the heterogeneous hardware units prior to the submission of the first media task exceeds a threshold; and
create the thread based on the latency time measure exceeding the threshold.
21. The at least one memory of claim 18 , wherein the media task is a first media task, and the instructions are to cause one or more of the at least one programmable circuit to:
determine a previous execution duration to process a second media task having a same type as the first media task at the selected one of the heterogeneous hardware units; and
set the estimated duration for the selected one of the heterogeneous hardware units based at least in part on the previous execution duration.