Managing artificial intelligence model partitions for edge computing environment
Techniques are disclosed for managing artificial intelligence model partitions for execution in an information processing system with edge computing resources. For example, a method comprises the following steps. An intermediate representation of an artificial intelligence model is obtained. A computation graph is generated based on the intermediate representation. The computation graph is partitioned into a set of partitions. The method then schedules the set of partitions for respective execution on a set of computing devices in an edge computing environment, and causes deployment of the set of partitions respectively to the set of computing devices for execution in the edge computing environment.
1. An apparatus, comprising:
a processor; and
a memory storing computer program instructions, the processor executing the computer program instructions in the memory to control the apparatus to:
obtain an intermediate representation of an artificial intelligence model;
obtain a computation graph generated based on the intermediate representation;
partition the computation graph into a set of partitions;
schedule the set of partitions for respective execution on a set of computing devices in an edge computing environment; and
cause deployment of the set of partitions respectively to the set of computing devices for execution in the edge computing environment;
wherein the obtaining, partitioning, scheduling, and causing deployment operations are performed in accordance with a model parallelism manager layer in coordination with an application manager layer; and
wherein the application manager layer is interposed between the model parallelism manager layer and the edge computing environment.
2. The apparatus of claim 1 , wherein causing deployment of the set of partitions to the set of computing devices further comprises a model scheduler dispatching a set of partition executers for execution to the set of computing devices in the edge computing environment.
3. The apparatus of claim 2 , wherein the model scheduler is configured as a remote procedure call server and the set of partition executers are configured as remote procedure call clients; and
wherein the model scheduler and the set of partition executers are implemented by the model parallelism manager layer.
4. The apparatus of claim 1 , wherein scheduling the set of partitions further comprises obtaining resource requirements for execution of the set of partitions in order to generate a scheduling plan.
5. The apparatus of claim 1 , wherein the processor further executes the computer program instructions in the memory to control the apparatus to compile each of the set of partitions to generate a runtime instance specific to a type of computing device on which each partition is to be executed.
6. The apparatus of claim 1 , wherein the processor further executes the computer program instructions in the memory to control the apparatus to re-schedule and cause re-deployment of one or more of the set of partitions when one or more of the computing devices in the edge computing environment change geographic locations.
7. The apparatus of claim 1 , wherein the intermediate representation of the artificial intelligence model is obtained from the artificial intelligence model written in a source language.
8. A method, comprising:
obtaining an intermediate representation of an artificial intelligence model;
obtaining a computation graph generated based on the intermediate representation;
partitioning the computation graph into a set of partitions;
scheduling the set of partitions for respective execution on a set of computing devices in an edge computing environment; and
causing deployment of the set of partitions respectively to the set of computing devices for execution in the edge computing environment;
wherein the method is performed by one or more processing devices;
wherein the obtaining, partitioning, scheduling and causing deployment are performed in accordance with a model parallelism manager layer in coordination with an application manager layer; and
wherein the application manager layer is interposed between the model parallelism manager layer and the edge computing environment.
9. The method of claim 8 , wherein causing deployment of the set of partitions to the set of computing devices further comprises dispatching, via a model scheduler, a set of partition executers for execution to the set of computing devices in the edge computing environment.
10. The method of claim 9 , wherein the model scheduler is configured as a remote procedure call server and the set of partition executers are configured as remote procedure call clients; and
wherein the model scheduler and the set of partition executers are implemented by the model parallelism manager layer.
11. The method of claim 8 , wherein scheduling the set of partitions further comprises obtaining resource requirements for execution of the set of partitions in order to generate a scheduling plan.
12. The method of claim 8 , further comprising compiling each of the set of partitions to generate a runtime instance specific to a type of computing device on which each partition is to be executed.
13. The method of claim 8 , further comprising re-scheduling and causing re-deployment of one or more of the set of partitions when one or more of the computing devices in the edge computing environment change geographic locations.
14. The method of claim 8 , wherein the intermediate representation of the artificial intelligence model is obtained from the artificial intelligence model written in a source language.
15. A computer program product stored on a non-transitory computer-readable medium and comprising machine executable instructions, the machine executable instructions, when executed, causing a processing device to:
obtain an intermediate representation of an artificial intelligence model;
obtain a computation graph generated based on the intermediate representation;
partition the computation graph into a set of partitions;
schedule the set of partitions for respective execution on a set of computing devices in an edge computing environment; and
cause deployment of the set of partitions respectively to the set of computing devices for execution in the edge computing environment;
wherein the obtaining, partitioning, scheduling and causing deployment are performed in accordance with a model parallelism manager layer in coordination with an application manager layer; and
wherein the application manager layer is interposed between the model parallelism manager layer and the edge computing environment.
16. The computer program product of claim 15 , wherein causing deployment of the set of partitions to the set of computing devices further comprises a model scheduler dispatching a set of partition executers for execution to the set of computing devices in the edge computing environment, wherein the model scheduler is configured as a remote procedure call server and the set of partition executers are configured as remote procedure call clients, and wherein the model scheduler and the set of partition executers are implemented by the model parallelism manager layer.
17. The computer program product of claim 15 , wherein scheduling the set of partitions further comprises obtaining resource requirements for execution of the set of partitions in order to generate a scheduling plan.
18. The computer program product of claim 15 , wherein the machine executable instructions, when executed, further cause the processing device to compile each of the set of partitions to generate a runtime instance specific to a type of computing device on which each partition is to be executed.
19. The computer program product of claim 15 , wherein the machine executable instructions, when executed, further cause the processing device to re-schedule and cause re-deployment of one or more of the set of partitions when one or more of the computing devices in the edge computing environment change geographic locations.
20. The computer program product of claim 15 , wherein the intermediate representation of the artificial intelligence model is obtained from the artificial intelligence model written in a source language.