Intelligent preemption system
Systems and methods herein describe an intelligent preemption system. The preemption system identifies a high priority task and a low priority task that are running on a processor, estimates a preemption save time and an execution time for the low priority task using a machine learning model trained to analyze historical preemption data, determines that the estimate preemption save time for the low priority task satisfies a first preemption condition and that the estimated execution time for the low priority task satisfies a second preemption condition, and in response to determining first preemption condition and second preemption condition are satisfied, scheduling a preemption event associated with the high priority task and the low priority task.
1 . A method comprising:
identifying, by a graphics processing unit (GPU), a high priority task and a low priority task that are running on the GPU;
monitoring, by the GPU, performance metrics of the low priority task;
estimating, by the GPU, a preemption save time and an execution time for the low priority task using a machine learning model trained to analyze historical preemption data;
based on the monitored performance metrics, determining, by the GPU, that the estimated preemption save time for the low priority task satisfies a first preemption condition, the first preemption condition comprising the estimated preemption save time for the low priority task exceeds the estimated execution time less a time until the low priority task is interrupted exceeding;
determining, by the GPU, that the estimated execution time for the low priority task satisfies a second preemption condition;
in response to determining that the estimated preemption save time satisfies the first preemption condition and determining that the estimated execution time for the low priority task satisfies the second preemption condition, scheduling, by the GPU, a preemption event associated with the high priority task and the low priority task; and
executing, by the GPU, the preemption event to switch execution from the low priority task to the high priority task on the GPU at a specified time.
2 . The method of claim 1 , wherein the machine learning model is a recurrent neural network.
3 . The method of claim 1 , further comprising:
estimating an execution time for the high priority task using the machine learning model.
4 . The method of claim 1 , further comprising:
receiving the preemption save time and the execution time for the low priority task as input from a user of a client device.
5 . The method of claim 1 , wherein the second preemption condition comprises a time until a deadline exceeding the estimated execution time less the time until the low priority task is interrupted plus an estimated execution time of the high priority task.
6 . The method of claim 1 , wherein the preemption save time and execution time for the low priority task are estimated based on at least one of: GPU clock cycles, memory usage, or task progress state.
7 . The method of claim 6 , wherein the historical preemption data comprises an average amount of time that the high priority task uses the GPU.
8 . A system comprising:
a graphics processing unit (GPU); and
a memory storing instructions that, when executed by the GPU, cause the system to:
identify, by the GPU a high priority task and a low priority task that are running on the GPU,
monitor, by the GPU, performance metrics of the low priority task;
estimate, by the GPU, a preemption save time and an execution time for the low priority task using a machine learning model trained to analyze historical preemption data;
based on the monitored performance metrics, etermine, by the GPU, that the estimated preemption save time for the low priority task satisfies a first preemption condition, the first preemption condition comprising the estimated preemption save time for the low priority task exceeds the estimated execution time less a time until the low priority task is interrupted exceeding;
determine, by the GPU, that the estimated execution time for the low priority task satisfies a second preemption condition;
in response to determining that the estimated preemption save time satisfies the first preemption condition and determining that the estimated execution time for the low priority task satisfies the second preemption condition, schedule, by the GPU, a preemption event associated with the high priority task and the low priority task; and
execute, by the GPU, the preemption event to switch execution from the low priority task to the high priority task on the GPU at a specified time .
9 . The system of claim 8 , wherein the machine learning model is a recurrent neural network.
10 . The system of claim 8 , wherein the instructions further configure the system to:
estimate an execution time for the high priority task using the machine learning model.
11 . The system of claim 8 , wherein the instructions further cause the system to:
receive the preemption save time and the execution time for the low priority task as input from a user of a client device.
12 . The system of claim 8 , wherein the second preemption condition comprises a time until a deadline exceeding the estimated execution time less the time until the low priority task is interrupted plus an estimated execution time of the high priority task.
13 . The system of claim 8 , wherein the preemption save time and execution time for the low priority task are estimated based on at least one of: GPU clock cycles, memory usage, or task progress state.
14 . The system of claim 13 , wherein the historical preemption data comprises an average amount of time that the high priority task uses the GPU.
15 . A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
identify, by a graphics processing unit (GPU), a high priority task and a low priority task that are running on the GPU;
monitor, by the GPU, performance metrics of the low priority task;
estimate, by the GPU, a preemption save time and an execution time for the low priority task using a machine learning model trained to analyze historical preemption data;
based on the monitored performance metrics, determine, by the GPU, that the estimated preemption save time for the low priority task satisfies a first preemption condition, the first preemption condition comprising the estimated preemption save time for the low priority task exceeds the estimated execution time less a time until the low priority task is interrupted exceeding;
determine, by the GPU, that the estimated execution time for the low priority task satisfies a second preemption condition;
in response to determining that the estimated preemption save time satisfies the first preemption condition and determining that the estimated execution time for the low priority task satisfies the second preemption condition, schedule, by the GPU, a preemption event associated with the high priority task and the low priority task; and
execute, by the GPU, the preemption event to switch execution from the low priority task to the high priority task on the GPU at a specified time.
16 . The computer-readable storage medium of claim 15 , wherein the machine learning model is a recurrent neural network.
17 . The computer-readable storage medium of claim 15 , wherein the instructions further cause the computer to:
estimate an execution time for the high priority task using the machine learning model.
18 . The computer-readable storage medium of claim 15 , wherein the instructions further cause the computer to:
receive the preemption save time and the execution time for the low priority task as input from a user of a client device.
19 . The computer-readable storage medium of claim 15 , wherein the second preemption condition comprises a time until a deadline exceeding the estimated execution time less the time until the low priority task is interrupted plus an estimated execution time of the high priority task.
20 . The computer-readable storage medium of claim 15 , wherein the preemption save time and execution time for the low priority task are estimated based on at least one of: GPU clock cycles, memory usage, or task progress state.