Persistent Multi-Instance GPU Partitions

1 . A method comprising:

providing a node comprising one or more applications;

providing a GPU;

dividing the GPU into one or more GPU instances, wherein each GPU instance is associated with at least one of the one or more applications;

saving partition data pertaining to the one or more GPU instances to a file; and

saving the file to a database.

2 . The method of claim 1 , further comprising:

rebooting the node, wherein rebooting the node deletes the one or more GPU instances;

accessing, by the node, a server when the node completes the reboot;

retrieving the file comprising the partition data;

creating new one or more GPU instances according to the partition data; and

associating the one or more applications with the new one or more GPU instances.

3 . The method of claim 2 , wherein dividing the GPU into one or more GPU instances, saving partition data pertaining to the one or more GPU instances to a file, and saving the file to a database are performed by an automated agent.

4 . The method of claim 3 , wherein retrieving the file comprising the partition data, creating new one or more partitions according to the partition data, and associating the one or more applications with the new one or more GPU instances are performed by the automated agent.

5 . The method of claim 1 , wherein the GPU is a plurality of GPUs and the plurality of GPUs each support one or more GPU instances, and wherein partition data of each GPU instance of each GPU is saved to the file.

6 . The method of claim 3 , wherein the node provides a handshake to the automated agent upon completing the reboot such that the automated agent receives an indication to retrieve the file from the server and partition the GPU.

7 . The method of claim 6 , wherein the automated agent saves the partition data to the file each time there is a change to a number of the GPU instances, a configuration of the GPU instances, or a mapping of the GPU instances to the one or more applications.

8 . The method of claim 1 , wherein the partition data is periodically saved to the file according to a time period and saved to the database, and wherein the time period is specified by a user.

9 . The method of claim 1 , wherein the partition data comprises one or more of a state of the GPU instances, a configuration of the GPU instances, metadata describing the GPU, or a ratio of the compute power in each GPU instance.

10 . The method of claim 2 , wherein accessing the server is performed by an automated agent, and wherein the automated agent provides a handshake to determine when the node finishes the reboot.

11 . A system comprising a memory and one or more processors configured to execute programming instructions stored on a non-transitory computer readable storage medium, where executing the programming instructions causes the system to:

provide a node comprising one or more applications;

provide a GPU;

divide the GPU into one or more GPU instances, wherein each GPU instance is associated with at least one of the one or more applications;

save partition data pertaining to the one or more GPU instances to a file; and

save the file to a database.

12 . The system of claim 11 , wherein executing the programming instructions further causes the system to:

reboot the node, wherein rebooting the node deletes the one or more GPU instances;

access, by the node, a server when the node completes the reboot;

retrieve the file comprising the partition data;

create new one or more GPU instances according to the partition data; and

associate the one or more applications with the new one or more GPU instances.

13 . The system of claim 12 , wherein dividing the GPU into one or more GPU instances, saving partition data pertaining to the one or more GPU instances to a file, and saving the file to a database are performed by an automated agent.

14 . The system of claim 13 , wherein retrieving the file comprising the partition data, creating new one or more partitions according to the partition data, and associating the one or more applications with the new one or more GPU instances are performed by the automated agent.

15 . The system of claim 11 , wherein the GPU is a plurality of GPUs and the plurality of GPUs each support one or more GPU instances, and wherein partition data of each GPU instance of each GPU is saved to the file.

16 . The system of claim 13 , wherein the node provides a handshake to the automated agent upon completing the reboot such that the automated agent receives an indication to retrieve the file from the server and partition the GPU.

17 . The system of claim 16 , wherein the automated agent saves the partition data to the file each time there is a change to a number of the GPU instances, a configuration of the GPU instances, or a mapping of the GPU instances.

18 . The system of claim 11 , wherein the partition data is periodically saved to the file according to a time period and saved to the database, and wherein the time period is specified by the user.

19 . The system of claim 12 , wherein accessing the server is performed by an automated agent, and wherein the automated agent provides a handshake to determine when the node comes back online.

20 . Non-transitory computer readable storage medium storing instructions for execution by one or more processors, the instructions comprising:

providing a node comprising one or more applications;

providing a GPU;

dividing the GPU into one or more GPU instances, wherein each GPU instance is associated with at least one of the one or more applications;

saving configuration data pertaining to the one or more GPU instances to a file;

saving the file to a database;

rebooting the node, wherein rebooting the node deletes the one or more GPU instances;

accessing, by the node, a server when the node completes the reboot;

retrieving the file comprising the configuration data;

creating new one or more GPU instances according to the configuration data; and

associating the one or more applications with the new one or more GPU instances.