SYSTEM AND METHOD FOR DYNAMIC SWITCHING OF GRAPHICS PROCESSING UNIT WORKLOADS
A system ( 108 ) and method ( 400 ) for dynamically managing graphics processing unit (GPU) workloads in GPU artificial intelligence (AI) cloud infrastructure ( 210 ) are disclosed. The method ( 400 ) involves monitoring, by an orchestrator ( 202 ), the GPU AI cloud infrastructure ( 210 ) comprising one or more types of workloads ( 208 ), wherein the one or more types of workloads ( 208 ) indicate different use cases that require computational tasks executed on the infrastructure. The orchestrator ( 202 ) receives one or more policy specifications from one or more users ( 102 ), wherein the policy specifications include a set of user-defined rules and configurations to manage the execution of the workloads ( 208 ) on one or more GPU resources. Based on the received policy specifications, the orchestrator ( 202 ) switches between the one or more types of workloads ( 208 ) and modifies the GPU AI cloud infrastructure ( 210 ) accordingly.
1 . A method ( 400 ) for dynamically switching graphics processing unit (GPU) workloads in GPU artificial intelligence (AI) cloud infrastructure ( 210 ), the method ( 400 ) comprising:
monitoring, by an orchestrator ( 202 ), the GPU AI cloud infrastructure ( 210 ) comprising one or more types of workloads ( 208 ), wherein the one or more types of workloads ( 208 ) indicate different use cases that require a computational tasks that are executed on the GPU AI cloud infrastructure ( 210 );
receiving, by the orchestrator ( 202 ), one or more policy specifications from a one or more users ( 102 ), wherein the one or more policy specifications indicate a set of user-defined rules and configurations to manage the one or more types of workloads ( 208 ) to run on one or more GPU resources;
switching, by the orchestrator ( 202 ), the one or more types of workloads based on the received one or more policy specifications; and
modifying, by the orchestrator ( 202 ), the GPU AI cloud infrastructure ( 210 ) based on the switching of the one or more types of workloads ( 208 ), thereby dynamically managing the GPU workloads in the GPU AI cloud infrastructure ( 210 ).
2 . The method ( 400 ) as claimed in claim 1 , wherein the monitoring the GPU AI cloud infrastructure ( 210 ), comprises:
tracking performance metrics of the one or more types of workloads ( 208 ) in real-time.
3 . The method ( 400 ) as claimed in claim 1 , comprising:
reallocating one or more GPU resources between the one or more types of workloads ( 208 ) based on the policy specified by the one or more users ( 102 ).
4 . The method ( 400 ) as claimed in claim 1 , wherein the one or more policy specifications include one or more pre-defined parameters related to the one or more types of workloads ( 208 ).
5 . A system ( 108 ) for dynamically switching graphics processing unit (GPU) workloads in a GPU AI cloud infrastructure ( 210 ), the system ( 108 ) comprising:
a memory ( 304 );
an orchestrator ( 202 ); and
at least one processor ( 302 ) in communication with the memory ( 304 ) and the orchestrator ( 202 ) is configured to:
monitor the GPU AI cloud infrastructure ( 210 ) comprising one or more types of workloads ( 208 ), wherein the one or more types of workloads ( 208 ) indicate different use cases that require computational tasks that are executed on the GPU AI cloud infrastructure ( 210 );
receive one or more policy specifications from a one or more users ( 102 ), wherein the one or more policy specifications indicate a set of user-defined rules and configurations to manage the one or more types of workloads ( 208 ) to run on one or more GPU resources;
switch the one or more types of workloads ( 208 ) based on the received one or more policy specifications; and
modify the GPU AI cloud infrastructure ( 210 ) based on the switching of the one or more types of workloads ( 208 ), thereby dynamically managing the GPU workloads in the GPU AI cloud infrastructure ( 210 ).
6 . The system ( 108 ) as claimed in claim 5 , wherein the monitoring the GPU AI cloud infrastructure ( 210 ), the at least one processor ( 302 ) is configured to:
track performance metrics of the one or more types of workloads ( 208 ) in real-time.
7 . The system ( 108 ) as claimed in claim 5 , wherein the at least one processor ( 302 ) is configured to:
reallocate one or more GPU resources between the one or more types of workloads ( 208 ) based on the one or more policy specifications received from the one or more users ( 102 ).
8 . The system ( 108 ) as claimed in claim 5 , wherein the one or more policy specifications include one or more pre-defined parameters related to the one or more types of workloads ( 208 ).