Intelligent data coordination for accelerated computing in cloud environment
Systems and methods are provided to implement intelligent data coordination for accelerated computing in a distributed computing environment. For example, a method includes executing a task on a computing node, monitoring requests issued by the executing task, intercepting requests issued by the executing task which correspond to data flow operations to be performed as part of the task execution, and asynchronously executing the intercepted requests at scheduled times to coordinate data flow between resources on the computing node.
1. A method, comprising:
performing an offline process which comprises:
executing a task;
determining data flow patterns which occur between resources as a result of data flow operations that are performed during the execution of the task; and
storing the determined data flow patterns in a knowledge base;
executing the task on a first computing node;
monitoring requests issued by the executing task;
intercepting requests issued by the executing task which correspond to data flow operations to be performed as part of the task execution, wherein the intercepted requests comprise at least one of requests for prefetching data from a memory, requests for loading data into a memory, requests for copying data from a first memory to a second memory, and requests for communicating data to a second computing node; and
asynchronously executing the intercepted requests at scheduled times to coordinate intra-node data flow between resources on the first computing node and inter-node data flow between resources on the first computing node and the second computing node, wherein asynchronously executing the intercepted requests at scheduled times comprises:
enqueuing the intercepted requests;
utilizing the determined data flow patterns in the knowledge base to schedule times for executing the enqueued requests in a manner which coordinates intra-node data flow between resources on the first computing node and inter-node data flow between resources on the first computing node and the second computing node; and
dispatching a given enqueued request for execution by an asynchronous background thread according to a scheduled time for the given enqueued request.
2. The method of claim 1 , wherein monitoring requests issued by the executing task comprises monitoring requests issued by the executing task to a software library and an application programming interface.
3. The method of claim 1 , wherein the intercepted requests comprise memory allocation requests.
4. The method of claim 1 , wherein asynchronously executing the intercepted requests at scheduled times further comprises:
scheduling the times to execute the intercepted requests based on contextual information associated with the intercepted requests and resource usage on at least one of the first computing node and the second computing node.
5. The method of claim 1 , wherein asynchronously executing the intercepted requests at scheduled times further comprises:
scheduling the times to execute the intercepted requests based on data flow patterns associated with the task, which are determined by performing a pilot run of the task.
6. The method of claim 1 , wherein the executing task comprises a machine learning task for training a machine learning model of a given type, and wherein the offline process is configured to (i) profile at least one of data locality and model parameters locality associated with the training of the machine learning model of the given type, and (ii) utilize at least one of the profiled data locality and the profiled model parameters locality to determine data flow patterns associated with the training of the machine learning model of the given type.
7. The method of claim 1 , wherein the intercepted requests comprise requests for copying data from a first memory of a first processor device to a second memory of a second processor device.
8. An article of manufacture comprising a processor-readable storage medium having stored program code of one or more software programs, wherein the program code is executable by one or more processors to implement method steps comprising:
performing an offline process which comprises:
executing a task;
determining data flow patterns which occur between resources as a result of data flow operations that are performed during the execution of the task; and
storing the determined data flow patterns in a knowledge base;
executing the task on a first computing node;
monitoring requests issued by the executing task;
intercepting requests issued by the executing task which correspond to data flow operations to be performed as part of the task execution, wherein the intercepted requests comprise at least one of requests for prefetching data from a memory, requests for loading data into a memory, requests for copying data from a first memory to a second memory, and requests for communicating data to a second computing node; and
asynchronously executing the intercepted requests at scheduled times to coordinate intra-node data flow between resources on the first computing node and inter-node data flow between resources on the first computing node and the second computing node, wherein asynchronously executing the intercepted requests at scheduled times comprises:
enqueuing the intercepted requests;
utilizing the determined data flow patterns in the knowledge base to schedule times for executing the enqueued requests in a manner which coordinates intra-node data flow between resources on the first computing node and inter-node data flow between resources on the first computing node and the second computing node; and
dispatching a given enqueued request for execution by an asynchronous background thread according to a scheduled time for the given enqueued request.
9. The article of manufacture of claim 8 , wherein monitoring requests issued by the executing task comprises monitoring requests issued by the executing task to a software library and an application programming interface.
10. The article of manufacture of claim 8 , wherein the intercepted requests comprise memory allocation requests.
11. The article of manufacture of claim 8 , wherein asynchronously executing the intercepted requests at scheduled times further comprises:
scheduling the times to execute the intercepted requests based on contextual information associated with the intercepted requests and resource usage on at least one of the first computing node and the second computing node.
12. The article of manufacture of claim 8 , wherein asynchronously executing the intercepted requests at scheduled times further comprises:
scheduling the times to execute the intercepted requests based on data flow patterns associated with the task, which are determined by performing a pilot run of the task.
13. The article of manufacture of claim 8 , wherein the executing task comprises a machine learning task for training a machine learning model of a given type, and wherein the offline process is configured to (i) profile at least one of data locality and model parameters locality associated with the training of the machine learning model of the given type, and (ii) utilize at least one of the profiled data locality and the profiled model parameters locality to determine data flow patterns associated with the training of the machine learning model of the given type.
14. The article of manufacture of claim 8 , wherein the intercepted requests comprise requests for copying data from a first memory of a first processor device to a second memory of a second processor device.
15. A system, comprising:
a memory configured to store program instruction; and
at least one processor configured to execute the stored program instructions to perform a method comprising:
performing an offline process which comprises:
executing a task;
determining data flow patterns which occur between resources as a result of data flow operations that are performed during the execution of the task; and
storing the determined data flow patterns in a knowledge base;
executing the task on a first computing node;
monitoring requests issued by the executing task;
intercepting requests issued by the executing task which correspond to data flow operations to be performed as part of the task execution, wherein the intercepted requests comprise at least one of requests for prefetching data from a memory, requests for loading data into a memory, requests for copying data from a first memory to a second memory, and requests for communicating data to a second computing node; and
asynchronously executing the intercepted requests at scheduled times to coordinate intra-node data flow between resources on the first computing node and inter-node data flow between resources on the first computing node and the second computing node, wherein asynchronously executing the intercepted requests at scheduled times comprises:
enqueuing the intercepted requests;
utilizing the determined data flow patterns in the knowledge base to schedule times for executing the enqueued requests in a manner which coordinates intra-node data flow between resources on the first computing node and inter-node data flow between resources on the first computing node and the second computing node; and
dispatching a given enqueued request for execution by an asynchronous background thread according to a scheduled time for the given enqueued request.
16. The system of claim 15 , wherein monitoring requests issued by the executing task comprises monitoring requests issued by the executing task to a software library and an application programming interface.
17. The system of claim 15 , wherein the intercepted requests comprise memory allocation requests.
18. The system of claim 15 , wherein asynchronously executing the intercepted requests at scheduled times further comprises:
scheduling the times to execute the intercepted requests based on contextual information associated with the intercepted requests and resource usage on at least one of the first computing node and the second computing node.
19. The system of claim 15 , wherein asynchronously executing the intercepted requests at scheduled times further comprises:
scheduling the times to execute the intercepted requests based on data flow patterns associated with the task, which are determined by performing a pilot run of the task.
20. The system of claim 15 , wherein the executing task comprises a machine learning task for training a machine learning model of a given type, and wherein the offline process is configured to (i) profile at least one of data locality and model parameters locality associated with the training of the machine learning model of the given type, and (ii) utilize at least one of the profiled data locality and the profiled model parameters locality to determine data flow patterns associated with the training of the machine learning model of the given type.