Simultaneous compute and graphics scheduling
A parallel processing unit (e.g., a GPU), in some examples, includes a hardware scheduler and hardware arbiter that launch graphics and compute work for simultaneous execution on a SIMD/SIMT processing unit. Each processing unit (e.g., a streaming multiprocessor) of the parallel processing unit operates in a graphics-greedy mode or a compute-greedy mode at respective times. The hardware arbiter, in response to a result of a comparison of at least one monitored performance or utilization metric to a user-configured threshold, can selectively cause the processing unit to run one or more compute work items from a compute queue when the processing unit is operating in the graphics-greedy mode, and cause the processing unit to run one or more graphics work items from a graphics queue when the processing unit is operating in the compute-greedy mode. Associated methods and systems are also described.
1. A processing system, comprising:
a Single Instruction Multiple Data (SIMD) or Single Instruction Multiple Thread (SIMT) processor that executes parallel instruction streams and is configured to operate in a graphics-greedy mode or a compute-greedy mode at respective times;
a hardware scheduler connected to the processor, the hardware scheduler scheduling the processor to simultaneously execute, in parallel, at least one graphics warp and at least one compute warp by selecting between (a) scheduling at least one compute warp to the processor while operating in the graphics-greedy mode repeatedly scheduling graphics warps to the processor from a graphics pipeline, and (b) scheduling at least one graphics warp to the processor while operating in the compute-greedy mode repeatedly scheduling compute warps to the processor from a compute pipeline; and
a hardware arbiter configured to, in response to a detected underutilization of a resource associated with the processor during said scheduling, determining a current operating mode of the processor and signaling the hardware scheduler to perform said scheduling at least one compute warp when the current operating mode is the graphics-greedy mode or said scheduling at least one graphics warp when the current operating mode is the compute-greedy mode.
2. A parallel processing unit, comprising:
a plurality of processing units, each processing unit configured to operate in a graphics-greedy mode or a compute-greedy mode at respective times, and to simultaneously run graphics work items from a graphics queue and compute work items from a compute queue;
a hardware scheduler configured to continuously select graphics work items from the graphics queue for running on a particular processing unit of the plurality of processing units when the particular processing unit is configured to operate in the graphics-greedy mode, and to continuously select compute work items from the compute queue for running on the particular processing unit when the particular processing unit is configured to operate in the compute-greedy mode; and
a hardware arbiter configured to, in response to a result of a comparison of at least one monitored performance or utilization metric to a user-configured threshold, selectively cause the particular processing unit to run one or more compute work items from the compute queue when the particular processing unit is configured to operate in the graphics-greedy mode, and to cause the particular processing unit to run one or more graphics work items from the graphics queue when the particular processing unit is configured to operate in the compute-greedy mode.
3. The parallel processing unit according to claim 2 , wherein each of the plurality of processing units is a single instruction multiple data (SIMD) processor or a single instruction multiple thread (SIMT) processor.
4. The parallel processing unit according to claim 2 , wherein the hardware scheduler is further configured to select the graphics-greedy mode or the compute-greedy mode based at least upon software-configured priority values associated with said graphics work items and compute work items.
5. The parallel processing unit according to claim 4 , wherein the hardware arbiter is further configured to select causing running of either compute work items or graphics work items based upon a software-configured scheduling policy.
6. The parallel processing unit according to claim 5 , wherein the hardware arbiter is further configured to select causing running of either compute work items or graphics work items based further upon occupancy metrics corresponding to occupancy of processing and memory resources by graphics work items and compute work items.
7. The parallel processing unit according to claim 6 , wherein the hardware arbiter is further configured to select causing running of either compute work items or graphics work items based upon metrics in said particular processing unit.
8. The parallel processing unit according to claim 6 , wherein the hardware arbiter is further configured to select causing running of either compute work items or graphics work items based upon metrics in said particular processing unit and other processing units.
9. The parallel processing unit according to claim 6 , wherein the hardware arbiter is further configured to select causing running of either compute work items or graphics work items based further upon output stalling metrics associated with graphics work items output from said processing unit and upon input starving metrics associated with input of graphics work items and compute work items to said processing unit.
10. The parallel processing unit according to claim 9 , wherein the hardware arbiter is further configured to select causing running of either compute work items or graphics work items based further upon time-averaged values of said occupancy metrics, said output stalling metrics associated with graphics work items and said input starving metrics associated with input of graphics work items and compute work items.
11. The parallel processing unit according to claim 10 , wherein the occupancy metrics comprises one or more of occupancy metrics for register files, occupancy metrics for warp resources, occupancy metrics for shared memory, and occupancy metrics for ISBE memory, wherein said input starving metrics associated with graphics work items comprising at least one of a vertex-associated queue and a pixel-associated queue, wherein said input starving metrics associated with compute work items comprising starving metrics associated with the compute queue, and wherein said output stalling metrics associated with graphics work items comprising output stalling metrics for at least one of a vertex-associated queue and a pixel-associated queue.
12. The parallel processing unit according to claim 9 , wherein the output stalling metrics include effects of back pressure from one or more fixed-function units processing graphics work items.
13. The parallel processing unit according to claim 4 , wherein the hardware scheduler or the hardware arbiter is further configured to determine a number of work items to be selected from the graphics queue or the compute queue based upon a respective trickle parameter specified in a software-specified policy.
14. The parallel processing unit according to claim 2 , wherein the hardware scheduler is further configured to
in response to determining to launch a group of graphics work items to the particular processing unit, launch one or more graphics work items already assigned to the hardware scheduler; and
in response to determining to launch a group of compute work items:
reserve resources associated with the particular processing unit for a particular number of compute work items; and
request the particular number of compute work items from the compute queue.
15. The parallel processing unit according to claim 14 , wherein the hardware scheduler is further configured to, in response to determining to launch the group of compute work items:
launch compute work items received in response to the requesting on the particular processing unit, or
not launch compute work items in response to receiving a negative acknowledgment to the requesting.
16. The parallel processing unit according to claim 15 , wherein the hardware scheduler is further configured to pass priority information to the particular processing unit with the launching of the graphics work items or the launching of the compute work items.
17. The parallel processing unit according to claim 2 , wherein the hardware scheduler is further configured to automatically change an active scheduling policy, and wherein the changing includes changing said user-configured threshold.
18. The parallel processing unit according to claim 17 , wherein the automatic changing is adaptive to workload.
19. The parallel processing unit according to claim 17 , wherein the automatic changing comprises asynchronously switching between multiple preconfigured scheduling profiles to change the active scheduling policy, each preconfigured scheduling profile including a respectively different set of scheduling policies.
20. The parallel processing unit according to claim 2 , wherein the hardware scheduler and/or hardware arbiter receive metrics from a plurality of processing units.
21. A method for performing graphics work items and compute work items simultaneously on a parallel processor having a plurality of processing units, comprising:
receiving the graphics work items from a graphics pipeline and the compute work items from a compute pipeline; and
scheduling a first group of said graphics work items and a second group of said compute work items to simultaneously execute on a selected single instruction multiple data (SIMD) or single instruction multiple thread (SIMT) processing unit of the plurality of processing units,
wherein the scheduling further comprises selecting the first group and the second group based further upon output stalling metrics associated with graphics work items output from said processing unit and upon input starving metrics associated with input of graphics work items and compute work items to said processing unit.
22. The method according to claim 21 , wherein the scheduling comprises selecting the first group and the second group based at least upon software-configured priority values associated with respective groups of said graphics work items and compute work items.
23. The method according to claim 22 , wherein the scheduling comprises selecting the first group and the second group based further upon a software-configured scheduling policy.
24. The method according to claim 22 , wherein the scheduling further comprises selecting the first group and the second group based further upon occupancy metrics corresponding to occupancy of processing and memory resources by graphics work items and compute work items.
25. The method according to claim 21 , wherein the selecting is further based upon time-averaged values of occupancy metrics corresponding to occupancy of processing and memory resources by graphics work items and compute work items, said output stalling metrics associated with graphics work items and said input starving metrics associated with input of graphics work items and compute work items.
26. A system, comprising:
a CPU configured to execute an application;
a memory configured to have a graphics queue to which graphics work items from the application are enqueued, and a compute queue to which compute work items from the application are enqueued; and
a graphics processing unit (GPU) comprising:
a plurality of processing units, each processing unit configured to operate in a graphics-greedy mode or a compute-greedy mode at respective times, and to simultaneously run graphics work items from the graphics queue and compute work items from the compute queue;
a hardware scheduler configured to continuously select graphics work items from the graphics queue for running on a particular processing unit of the plurality of processing units when the particular processing unit is configured to operate in the graphics-greedy mode, and to continuously select compute work items from the compute queue for running on the particular processing unit when the particular processing unit is configured to operate in the compute-greedy mode; and
a hardware arbiter configured to, in response to a result of a comparison of at least one monitored performance or utilization metric to a user-configured threshold, selectively cause the particular processing unit to run one or more compute work items from the compute queue when the particular processing unit is configured to operate in the graphics-greedy mode, and to cause the particular processing unit to run one or more graphics work items from the graphics queue when the particular processing unit is configured to operate in the compute-greedy mode.