Machine learning based power and performance optimization system and method for graphics processing units
Systems and methods to dynamically adjust operating conditions of a graphics processing unit (GPU) are disclosed. A machine learning model is trained to determine operating voltages and frequencies to be provided to a GPU core of the GPU to execute a workload comprising a plurality of commands. The trained machine learning model is deployed to firmware of the GPU. A command in the workload to be executed by the GPU core is received. The trained machine learning model determines operating voltage and frequency for the GPU core to execute the command.
1. A computer-implemented method to dynamically adjust operating conditions of a graphics processing unit (GPU), the method comprising:
training a machine learning model to determine operating voltages and frequencies to be provided to a GPU core of the GPU to execute a workload comprising a plurality of commands;
deploying the trained machine learning model to firmware of the GPU;
receiving a command in the workload to be executed by the GPU core; and
determining, by the trained machine learning model and based on the command, operating voltage and frequency for the GPU core to execute the command.
2. The computer-implemented method of claim 1 , further comprising:
providing the determined operating voltage and frequency to the GPU core before the GPU core executes the command.
3. The computer-implemented method of claim 1 , wherein the machine learning model comprises a voltage-frequency inference model trained to determine operating voltages and frequencies of the GPU core in executing commands.
4. The computer-implemented method of claim 3 , wherein the machine learning model further comprises a draw performance inference model trained to determine performance data of the GPU core in executing the commands.
5. The computer-implemented method of claim 4 , wherein the machine learning model further comprises a power inference model trained to determine power consumptions of the GPU core in executing the commands.
6. The computer-implemented method of claim 5 , further comprising:
adjusting the operating voltage and frequency for the GPU core if a determined power consumption exceeds a threshold.
7. The computer-implemented method of claim 5 , wherein determining the operating voltage and frequency for the GPU core to execute the command comprises:
determining, based on the command, performance data of the GPU core in executing the command;
determining, based on the command, a power consumption of the GPU core in executing the command; and
generating, based on the command, the determined performance data, and the determined power consumption, an optimal operating voltage and frequency for the GPU core to execute the command.
8. The computer-implemented method of claim 1 , further comprising:
training the machine learning model with a first dataset generated from a GPU simulation model, wherein the first dataset comprises data associated with simulated commands for a workload, simulated operating voltages, simulated operating frequencies, simulated power consumption and simulated performance data of the GPU core in executing the simulated commands.
9. The computer-implemented method of claim 8 , further comprising:
training the machine learning model with a second dataset generated from the GPU simulation model, wherein the second dataset comprises data associated with simulated commands for a workload, simulated operating voltages, simulated operating frequencies, simulated power consumption and simulated performance data of the GPU core in executing the simulated commands.
10. The computer-implemented method of claim 1 , further comprising:
receiving a second command to be executed by the GPU core; and
determining, by the trained machine learning model, based on the second command, second operating voltage and frequency for the GPU core to execute the second command.
11. The computer-implemented method of claim 1 , wherein the commands of the workload include at least a draw command for rendering three-dimensional graphics.
12. The computer-implemented method of claim 1 , wherein the trained machine learning model is not based on a feedback mechanism in which the trained machine learning model provides operating frequency and voltage to the GPU core to execute a current workload based on operating frequency and voltage of the GPU core in executing a previous workload.
13. A non-volatile memory of a computing system storing instructions that, when executed, cause the computing system to perform a method of dynamically adjusting operating conditions of a graphics processing unit (GPU), the method comprising:
receiving a command in a workload to be executed by a GPU core of the GPU; and
determining, by a machine learning model and based on the command, operating voltage and frequency for the GPU core to execute the command.
14. The non-volatile memory of claim 13 , wherein the method further comprising:
providing the determined operating voltage and frequency to the GPU core before the GPU core executes the command.
15. The non-volatile memory of claim 13 , wherein the machine learning model comprises a voltage-frequency inference model trained to determine operating voltages and frequencies of the GPU core in executing commands.
16. The non-volatile memory of claim 15 , wherein the machine learning model further comprises a draw performance inference model trained to determine performance data of the GPU core in executing the commands.
17. The non-volatile memory of claim 16 , wherein the machine learning model further comprises a power inference model trained to determine power consumptions of the GPU core in executing the commands.
18. The non-volatile memory of claim 17 , wherein the instructions, when executed, caused the computing system to further perform:
adjusting the operating voltage and frequency for the GPU core if the determined power consumption exceeds a threshold.
19. The non-volatile memory of claim 17 , wherein determining the operating voltage and frequency for the GPU core to execute the command comprises:
determining, based on the command, performance data of the GPU core in executing the command;
determining, based on the command, a power consumption of the GPU core in executing the command; and
generating, based on the command, the determined performance data, and the determined power consumption, an optimal operating voltage and frequency for the GPU core to execute the command.
20. The non-volatile memory of claim 13 , wherein commands of the workload include at least a draw command for rendering three-dimensional graphics.