Orchestration of workloads involving an AI model
The present disclosure relates to a method comprising receiving a request to execute a workload using an artificial intelligence model. A current resource utilization status in the distributed system may be determined. The current resource utilization status may be used to define a deployment configuration of the artificial intelligence model, wherein the deployment configuration is defined by: a number and structure of input blocks, a number and structure of output blocks and the intermediate block of the artificial intelligence model, a second computer system to execute the intermediate block, and one or more first computer systems to execute the input and output blocks. The artificial intelligence model may be deployed in accordance with the defined deployment configuration and the workload may be executed.
1. A method for executing workloads in a distributed system using an artificial intelligence model, comprising:
the distributed system comprising a set of first computer systems configured to connect to at least one second computer system of the distributed system,
the artificial intelligence model configured to receive a specific input, process the specific input and provide a specific output, the artificial intelligence model further configured to be split into a set of one or more input blocks, an intermediate block and a set of one or more output blocks, such that the set of one or more input blocks receive the specific input and provides an intermediate output, the intermediate block receives as input the intermediate output and provides a second intermediate output, and the set of one or more output blocks receive as input the second intermediate output and provides said specific output;
the method further comprising an orchestration method comprising:
receiving a request to execute a workload using the artificial intelligence model, the workload comprising receiving the specific input;
determining a current resource utilization status in the distributed system;
using the current resource utilization status to define a deployment configuration of the artificial intelligence model, wherein the deployment configuration is defined by:
the artificial intelligence model comprising a number and structure of input blocks, a number and structure of output blocks and the intermediate block of,
a second computer system to execute the intermediate block, and
one or more first computer systems to execute the number and structure of input blocks and the number and structure of output blocks;
wherein defining the deployment configuration further comprises identifying whether an existing deployment configuration is present based on a previous deployment of the artificial intelligence model;
based on the presence of the existing deployment configuration, determining whether the existing deployment configuration is valid;
based on determining that the existing deployment configuration is valid, using the existing deployment configuration as the defined deployment configuration;
based on determining that the existing deployment configuration is not valid, changing the existing deployment configuration and using the changed existing deployment configuration as the defined deployment configuration, wherein the changing further comprises changing at least one of the one or more first computer systems, the second computer system, the number and structure of the input blocks, the number and structure of the output blocks, and the intermediate block associated with the existing deployment configuration; and
deploying the artificial intelligence model in accordance with the defined deployment configuration and executing the workload.
2. The method of claim 1 , wherein before executing the orchestration method, the method comprises deploying the artificial intelligence model in accordance with an initial deployment configuration, wherein defining the deployment configuration comprises updating the initial deployment configuration, the updating comprising at least one of:
re-splitting the artificial intelligence model;
adding the one or more first computer systems to execute the number and structure of input blocks and the number and structure of output blocks;
removing the one or more of the first computer systems of the initial deployment configuration to execute the number and structure of input blocks and the number and structure of output blocks; or
selecting the at least one second computer system to execute the intermediate block.
3. The method of claim 1 , wherein using the current resource utilization status to define the deployment configuration comprises:
performing a capacity profiling of the one or more first computer systems and the second computer system to determine whether each of the one or more first computer systems and the at least one second computer system can execute one or more blocks of the artificial intelligence model;
and defining the deployment configuration based on the capacity profiling.
4. The method of claim 1 , wherein the current resource utilization status is defined by at least one of:
utilization level of network resources of the distributed system;
utilization level of resources of the one or more first computer systems; or
utilization level of resources of the at least one second computer systems.
5. The method of claim 1 , wherein the deployment configuration is defined further using a rule, the rule requiring at least one of:
a maximum number of the one or more first computer systems to be used for deploying the number and structure of input blocks and the number and structure of output blocks;
a maximum number of blocks of the artificial intelligence model;
a fulfilment of a specific Quality of Service (QOS) requirement; or
a desired energy-efficiency (EE).
6. The method of claim 1 , wherein the distributed system includes a wireless communication system, wherein the one or more first computer systems are multi-access edge computing (MEC) nodes, and wherein the at least one second computer system is a cloud system.
7. The method of claim 1 , further comprising:
executing the deployment of the artificial intelligence model by a broadcaster of the distributed system;
sending by the broadcaster an information to an orchestrator of the distributed system, such that the orchestrator triggers the execution of the workload upon receiving the information.
8. The method of claim 1 , wherein the artificial intelligence model is a foundation model.
9. The method of claim 1 , wherein the artificial intelligence model is a deep neural network or a transformer, wherein the number and structure of input blocks represents first network layers, the intermediate block represents middle network layers; and the number and structure of output blocks represents last network layers.
10. The method of claim 1 , wherein the one or more first computer systems has an amount of processing resources which is smaller than other processing resources of the at least one second computer system.
11. The method of claim 1 , wherein the workload involves at least one of: data analytics,
sensor measurement fusion from different sources, image analysis, or processing data streams destined to a cloud.
12. The method of claim 1 , wherein the executing of the artificial intelligence model further comprises:
for each two consecutive blocks of the artificial intelligence model which are deployed on different computer systems, encoding using an encoding protocol the output of a first block of the two blocks, and sending the encoded output to a first computer system or to the at least one second computer system in order to be used as input for a second block of the two blocks.
13. The method of claim 12 , wherein the encoding protocol comprises at least one of compression or encryption.
14. A computer program product comprising a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configured to execute a method comprising:
receiving a request to execute a workload using an artificial intelligence model, the workload comprising receiving a specific input;
determining a current resource utilization status in a distributed system;
using the current resource utilization status to define a deployment configuration of the artificial intelligence model, wherein the deployment configuration is defined by:
a number and structure of input blocks, a number and structure of output blocks, and an intermediate block of the artificial intelligence model,
a second computer system to execute the intermediate block, and
one or more first computer systems to execute the number and structure of input blocks and the number and structure of output blocks;
wherein defining the deployment configuration further comprises identifying whether an existing deployment configuration is present based on a previous deployment of the artificial intelligence model;
based on the presence of the existing deployment configuration, determining whether the existing deployment configuration is valid;
based on determining that the existing deployment configuration is valid, using the existing deployment configuration as the defined deployment configuration;
based on determining that the existing deployment configuration is not valid, changing the existing deployment configuration and using the changed existing deployment configuration as the defined deployment configuration, wherein the changing further comprises changing at least one of the one or more first computer systems, the second computer system, the number and structure of the input blocks, the number and structure of the output blocks, and the intermediate block associated with the existing deployment configuration; and
deploying the artificial intelligence model in accordance with the defined deployment configuration and executing the workload.
15. A computer system for executing workloads in a distributed system using an artificial intelligence model, comprising:
the distributed system comprising a set of first computer systems which are configured to connect to at least one second computer system of the distributed system,
the artificial intelligence model configured to receive a specific input, process the specific input and provide a specific output, the artificial intelligence model being configured to be split into a set of one or more input blocks, an intermediate block and a set of one or more output blocks, such that the set of one or more input blocks receive the specific input and provides an intermediate output, the intermediate block receives as input the intermediate output and provides a second intermediate output, and the set of one or more output blocks receive as input the second intermediate output and provides said specific output;
the computer system comprising computer program instructions for:
receiving a request to execute a workload using the artificial intelligence model, the workload comprising receiving the specific input;
determining a current resource utilization status in the distributed system;
using the current resource utilization status to define a deployment configuration of the artificial intelligence model, wherein the deployment configuration is defined by:
a number and structure of input blocks, a number and structure of output blocks, and the intermediate block of the artificial intelligence model,
a second computer system to execute the intermediate block, and
one or more first computer systems to execute the number and structure of input blocks and the number and structure of output blocks;
wherein defining the deployment configuration further comprises identifying whether an existing deployment configuration is present based on a previous deployment of the artificial intelligence model;
based on the presence of the existing deployment configuration, determining whether the existing deployment configuration is valid;
based on determining that the existing deployment configuration is valid, using the existing deployment configuration as the defined deployment configuration;
based on determining that the existing deployment configuration is not valid, changing the existing deployment configuration and using the changed existing deployment configuration as the defined deployment configuration, wherein the changing further comprises changing at least one of the one or more first computer systems, the second computer system, the number and structure of the input blocks, the number and structure of the output blocks, and the intermediate block associated with the existing deployment configuration; and
deploying the artificial intelligence model in accordance with the defined deployment configuration and executing the workload.