Checkpointing for GPU-as-a-service in cloud computing environment
Techniques are provided to configure a graphics processing unit (GPU) service platform to provide GPU processing and checkpointing services to a client system. For example, when the GPU service platform receives a service request from a first client system for GPU processing services, a GPU server node is allocated to execute a first GPU processing task associated with the service request using one or more GPU devices. When a GPU checkpointing operation is performed during execution of the first GPU processing task to generate a checkpoint image of a current state of the first GPU processing task, the GPU server node is configured to execute at least a portion of a second GPU processing task associated with a second service request received from a second client system, using the one or more GPU devices concurrently with performing the GPU checkpointing operation for the first GPU processing task.
1. A method, comprising:
receiving, by a Graphics Processing Unit-as-a-Service (GPUaaS) cloud computing platform, a first service request from a first client system for GPU processing services provided by the GPUaaS cloud computing platform, wherein the GPUaaS cloud computing platform comprises a GPU server node and GPU checkpoint service controller, wherein the GPU server node comprises one or more GPU devices;
executing, by the GPU server node, a first GPU processing task associated with the first service request received from the first client system using the one or more GPU devices of the GPU server node;
receiving, by the GPUaaS cloud computing platform, a second service request from a second client system for GPU processing services provided by the GPUaaS cloud computing platform;
executing, by the GPU server node, a second GPU processing task associated with the second service request received from the second client system, using the one or more GPU devices of the GPU server node; and
performing, by the GPU checkpoint service controller, a GPU checkpointing operation for the first client system during execution of the first GPU processing task to generate a checkpoint image of a current state of the first GPU processing task;
wherein the GPU server node continues executing the second GPU processing task using the one or more GPU devices concurrently with the GPU checkpoint service controller performing the GPU checkpointing operation for the first GPU processing task executing on the one or more GPU devices.
2. The method of claim 1 , wherein performing the GPU checkpointing operation during execution of the first GPU processing task comprises:
detecting, by the GPU server node, an occurrence of a predefined event in the execution of the first GPU processing task; and
automatically initiating, by the GPU checkpoint service controller, the GPU checkpointing operation in response to the detected occurrence of the predefined event.
3. The method of claim 2 , further comprising:
recording, by the GPU server node, application programming interface (API) messages transmitted between the GPU server node and the first client system;
wherein performing the GPU checkpointing operation comprises:
generating a checkpoint image of in-memory data stored in a memory device of the GPU server node, wherein the in-memory data represents the current state of the first GPU processing task; and
storing the checkpoint image along with a copy of the recorded API messages exchanged between the first client system and the GPU server node in a data storage node.
4. The method of claim 2 , wherein the predefined event comprises detecting a request from the first client system to send a copy of the current state of GPU memory from the GPU server node to a processing unit of the first client system.
5. The method of claim 2 , wherein the predefined event comprises an expiration of a predetermined time interval for performing a checkpoint operation.
6. The method of claim 1 , wherein performing the GPU checkpointing operation during execution of the first GPU processing task comprises:
receiving, by the GPU server node, an explicit checkpoint command from the first client system;
sending, by the GPU server node, a checkpointing tag to the first client system, which is utilized to identify a checkpoint image associated with the explicit checkpoint command; and
scheduling, by the GPU server node, a time for commencing the GPU checkpointing operation.
7. The method of claim 6 , wherein scheduling the time for commencing the GPU checkpointing operation comprises:
storing, by the GPU server node, a checkpointing task in a global task queue maintained by the GPU server node;
dispatching, by the GPU server node, the checkpointing task at the scheduled time for commencing the GPU checkpointing operation; and
preventing, by the GPU server node, execution of other GPU processing tasks of the first client system on the one or more GPU device of the GPU server node during execution of the GPU checkpointing operation.
8. A method, comprising:
receiving, by a graphics processing unit (GPU) service platform, a first service request from a first client system for GPU processing services provided by the GPU service platform, wherein the GPU service platform comprises a GPU server node, wherein the GPU server node comprises one or more GPU devices;
executing, by the GPU server node, a first GPU processing task associated with the first service request using the one or more GPU devices;
performing, by the GPU server node, a GPU checkpointing operation during execution of the first GPU processing task to generate a checkpoint image of a current state of the first GPU processing task; and
executing, by the GPU server node, a second GPU processing task associated with a second service request received from a second client system, using the one or more GPU devices;
wherein at least a portion of the second GPU processing task is executed by the GPU server node concurrently with performing the GPU checkpointing operation for the first GPU processing task;
wherein executing the first GPU processing task associated with the first service request comprises allocating, by the GPU service platform, at least two GPU server nodes within a cluster of GPU server nodes to handle execution of the first GPU processing task; and
wherein performing the GPU checkpointing operation comprises each of the at least two GPU server nodes performing a local checkpointing operation to generate a local checkpoint image of a current state of a respective portion of the first GPU processing task executing on that GPU server node, and combining the local checkpoint images to generate a complete checkpoint image of the current state of the first GPU processing task.
9. The method of claim 8 , further comprising:
storing, by the GPU service platform, an additional service request received from the first client system in a queue pending completion of the GPU checkpointing operation for the first client system; and
sending, by the GPU service platform, the additional service request received from the first client system to one or more GPU server nodes that are allocated to handle the additional service request.
10. An article of manufacture comprising a processor-readable storage medium having stored program code of one or more software programs, wherein the program code is executable by one or more processors to implement method steps comprising:
receiving, by a Graphics Processing Unit-as-a-Service (GPUaaS) cloud computing platform, a first service request from a first client system for GPU processing services provided by the GPUaaS cloud computing platform, wherein the GPUaaS cloud computing platform comprises a GPU server node and GPU checkpoint service controller, wherein the GPU server node comprises one or more GPU devices;
executing, by the GPU server node, a first GPU processing task associated with the first service request received from the first client system using the one or more GPU devices of the GPU server node;
receiving, by the GPUaaS cloud computing platform, a second service request from a second client system for GPU processing services provided by the GPUaaS cloud computing platform;
executing, by the GPU server node, a second GPU processing task associated with the second service request received from the second client system, using the one or more GPU devices of the GPU server node; and
performing, by the GPU checkpoint service controller, a GPU checkpointing operation for the first client system during execution of the first GPU processing task to generate a checkpoint image of a current state of the first GPU processing task;
wherein the GPU server node continues executing the second GPU processing task using the one or more GPU devices concurrently with the GPU checkpoint service controller performing the GPU checkpointing operation for the first GPU processing task executing on the one or more GPU devices.
11. The article of manufacture of claim 10 , wherein performing the GPU checkpointing operation during execution of the first GPU processing task comprises:
detecting, by the GPU server node, an occurrence of a predefined event in the execution of the first GPU processing task; and
automatically initiating, by the GPU checkpoint service controller, the GPU checkpointing operation in response to the detected occurrence of the predefined event.
12. The article of manufacture of claim 11 , further comprising program code that is executable by the one or more processors to perform methods step of:
recording, by the GPU server node, application programming interface (API) messages transmitted between the GPU server node and the first client system;
wherein performing the GPU checkpointing operation comprises:
generating a checkpoint image of in-memory data stored in a memory device of the GPU server node, wherein the in-memory data represents the current state of the first GPU processing task; and
storing the checkpoint image along with a copy of the recorded API messages exchanged between the first client system and the GPU server node in a data storage node.
13. The article of manufacture of claim 11 , wherein the predefined event comprises at least one of (i) detecting a request from the first client system to send a copy of the current state of GPU memory from the GPU server node to a processing unit of the first client system and an expiration of a predetermined time interval for performing a checkpoint operation.
14. The article of manufacture of claim 10 , wherein performing the GPU checkpointing operation during execution of the first GPU processing task comprises:
receiving, by the GPU server node, an explicit checkpoint command from the first client system;
sending, by the GPU server node, a checkpointing tag to the first client system, which is utilized to identify a checkpoint image associated with the explicit checkpoint command; and
scheduling, by the GPU server node, a time for commencing the GPU checkpointing operation.
15. The article of manufacture of claim 14 , wherein scheduling the time for commencing the GPU checkpointing operation comprises:
storing, by the GPU server node, a checkpointing task in a global task queue maintained by the GPU server node;
dispatching, by the GPU server node, the checkpointing task at the scheduled time for commencing the GPU checkpointing operation; and
preventing, by the GPU server node, execution of other GPU processing tasks of the first client system on the one or more GPU device of the GPU server node during execution of the GPU checkpointing operation.
16. The article of manufacture of claim 10 , wherein:
executing the first GPU processing task associated with the first service request comprises allocating, by the GPU service platform, at least two GPU server nodes within a cluster of GPU server nodes to handle execution of the first GPU processing task; and
performing the GPU checkpointing operation comprises each of the at least two GPU server nodes performing a local checkpointing operation to generate a local checkpoint image of a current state of a respective portion of the first GPU processing task executing on that GPU server node, and combining the local checkpoint images to generate a complete checkpoint image of the current state of the first GPU processing task.
17. The article of manufacture of claim 16 , further comprising program code that is executable by the one or more processors to perform method steps of:
storing, by the GPU service platform, an additional service request received from the first client system in a queue pending completion of the GPU checkpointing operation for the first client system; and
sending, by the GPU service platform, the additional service request received from the first client system to one or more GPU server nodes that are allocated to handle the additional service request.
18. A Graphics Processing Unit-as-a-Service (GPUaaS) cloud computing platform, comprising:
a GPU checkpoint service controller;
a GPU server node comprising one or more GPU devices;
a memory configured to store program instruction; and
at least one processor configured to execute the stored program instructions to perform a method comprising:
receiving, by the GPUaaS cloud computing platform, a first service request from a first client system for GPU processing services provided by the GPUaaS cloud computing platform;
executing, by the GPU server node, a first GPU processing task associated with the first service request received from the first client system using the one or more GPU devices;
receiving, by the GPUaaS cloud computing platform, a second service request from a second client system for GPU processing services provided by the GPUaaS cloud computing platform;
executing, by the GPU server node, a second GPU processing task associated with the second service request received from the second client system, using the one or more GPU devices of the GPU server node; and
performing, by the GPU checkpoint service controller, a GPU checkpointing operation during execution of the first GPU processing task to generate a checkpoint image of a current state of the first GPU processing task;
wherein the GPU server node continues executing the second GPU processing task using the one or more GPU devices concurrently with the GPU checkpoint service controller performing the GPU checkpointing operation for the first GPU processing task executing on the one or more GPU devices.
19. The GPUaaS cloud computing platform of claim 18 , wherein performing the GPU checkpointing operation during execution of the first GPU processing task comprises:
recording application programming interface (API) messages transmitted between the GPU server node and the first client system;
detecting an occurrence of a predefined event in the execution of the first GPU processing task; and
automatically initiating the GPU checkpointing operation by the GPU checkpoint service controller in response to the detected occurrence of the predefined event;
wherein performing the GPU checkpointing operation comprises:
generating a checkpoint image of in-memory data stored in a memory device of the GPU server node, wherein the in-memory data represents the current state of the first GPU processing task; and
storing the checkpoint image along with a copy of the recorded API messages exchanged between the first client system and the GPU server node in a data storage node.
20. The GPUaaS cloud computing platform of claim 18 , wherein performing the GPU checkpointing operation during execution of the first GPU processing task comprises:
receiving an explicit checkpoint command from the first client system;
sending a checkpointing tag to the first client system, which is utilized to identify a checkpoint image associated with the explicit checkpoint command; and
scheduling a time for commencing the GPU checkpointing operation, wherein scheduling the time for commencing the GPU checkpointing operation comprises:
storing a checkpointing task in a global task queue maintained by the GPU server node;
dispatching the checkpointing task at the scheduled time for commencing the GPU checkpointing operation; and
preventing execution of other GPU processing tasks of the first client system on the one or more GPU device of the GPU server node during execution of the GPU checkpointing operation.