System and method for distributed graphics processing unit (GPU) computation

1. A system comprising:

a data processor; and

a distributed task management module, executable by the data processor, the distributed task management module being configured to:

receive a user task service request from a user node;

query resource availability from a plurality of slave nodes having a plurality of graphics processing units (GPUs) thereon, the plurality of slave nodes configured with multiple GPUs mounted on distributed processing containers;

generate a list of uniform resource locators (URLs), each URL on the list corresponding to a path to an available distributed processing container on the plurality of slave nodes;

issue the list of URLs to a load balancing node;

receive from the load balancing node an overall unique URL corresponding to the list of URLs;

use the overall unique URL to assign the user task service request to a plurality of available GPUs based on the resource availability and resource requirements of the user task service request, the assigning including using available distributed processing containers on the plurality of slave nodes; and

retain the list of URLs corresponding to the distributed processing containers assigned to the user task service request.

2. The system of claim 1 wherein the distributed processing containers are implemented using a container-enabled system having a plurality of distributed containers for processing data.

3. The system of claim 1 wherein the distributed task management module being configured to forward the overall unique URL to a user node that originated the user task service request.

4. The system of claim 1 wherein the distributed task management module being configured to determine a number of GPUs mounted within an individual slave node and a number of resources available therein.

5. The system of claim 1 wherein assigning the user task service request is based on whether the user task service request is a GPU intensive task or a central processing unit (CPU) intensive task.

6. A method comprising:

receiving a user task service request from a user node;

querying resource availability from a plurality of slave nodes having a plurality of graphics processing units (GPUs) thereon, the plurality of slave nodes configured with multiple GPUs mounted on distributed processing containers;

generating a list of uniform resource locators (URLs), each URL on the list corresponding to a path to an available distributed processing container on the plurality of slave nodes;

issuing the list of URLs to a load balancing node;

receiving from the load balancing node an overall unique URL corresponding to the list of URLs;

using the overall unique URL to assign the user task service request to a plurality of available GPUs based on the resource availability and resource requirements of the user task service request, the assigning including using available distributed processing containers on the plurality of slave nodes; and

retaining the list of URLs corresponding to the distributed processing containers assigned to the user task service request.

7. The method of claim 6 wherein the distributed processing containers are implemented using a container-enabled system having a plurality of distributed containers for processing data.

8. The method of claim 6 including forwarding the overall unique URL to a user node that originated the user task service request.

9. The method of claim 6 including determining a number of GPUs mounted within an individual slave node and a number of resources available therein.

10. The method of claim 6 wherein assigning the user task service request is based on whether the user task service request is a GPU intensive task or a central processing unit (CPU) intensive task.

11. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

receive a user task service request from a user node;

query resource availability from a plurality of slave nodes having a plurality of graphics processing units (GPUs) thereon, the plurality of slave nodes configured with multiple GPUs mounted on distributed processing containers;

generate a list of uniform resource locators (URLs), each URL on the list corresponding to a path to an available distributed processing container on the plurality of slave nodes;

issue the list of URLs to a load balancing node;

receive from the load balancing node an overall unique URL corresponding to the list of URLs;

use the overall unique URL to assign the user task service request to a plurality of available GPUs based on the resource availability and resource requirements of the user task service request, the assigning including using available distributed processing containers on the plurality of slave nodes; and

retain the list of URLs corresponding to the distributed processing containers assigned to the user task service request.

12. The machine-useable storage medium of claim 11 wherein the distributed processing containers are implemented using a container-enabled system having a plurality of distributed containers for processing data.

13. The machine-useable storage medium of claim 11 wherein the instructions being configured forward the overall unique URL to a user node that originated the user task service request.

14. The machine-useable storage medium of claim 11 wherein assigning the user task service request is based on whether the user task service request is a GPU intensive task or a central processing unit (CPU) intensive task.