Distributed execution of an artificial intelligence model
The present disclosure relates to a method for executing an artificial intelligence model, including receiving an input for execution of the AI model. An input block of the AI model can be executed by a first computer system using the received input, producing a first output. The first output can be encoded. The encoded first output can be sent to a second computer system. The second computer system can decode the encoded first output. The second computer system can execute an intermediate block of the AI model using the first output, producing a second output. The second output can be encoded. The encoded second output can be sent to the first computer system. The first computer system can decode the encoded second output. The first computer system can execute an output block of the ML model using as input the second output, producing a result output.
1 . A computer-implemented method for executing an artificial intelligence model, comprising:
receiving an input for execution of the artificial intelligence model, wherein the artificial intelligence model is split into an input block, an intermediate block, and an output block, such that the input block receives specific input and provides an intermediate output, the intermediate block receives as input the intermediate output and provides another intermediate output, and the output block receives as input the another intermediate output and provides specific output;
executing the input block by a first computer system using the input, producing a first intermediate output;
encoding by the first computer system the first intermediate output using a first encoding protocol to produce an encoded first intermediate output;
sending the encoded first intermediate output to a second computer system to allow the second computer system to:
decode the encoded first intermediate output using the first encoding protocol, and execute the intermediate block using as input the first intermediate output to produce a second intermediate output,
encode the second intermediate output using a second encoding protocol to produce an encoded second intermediate output, and send the encoded second intermediate output to the first computer system; and
in response to receiving the encoded second intermediate output, decoding by the first computer system the encoded second intermediate output using the second encoding protocol, and executing the output block at the first computer system using as input the second intermediate output, producing a result output of a task performed by the artificial intelligence model as a whole by way of the input block, intermediate block, and output block.
2 . The computer-implemented method of claim 1 , further comprising: before executing the output block, deleting the input block from the first computer system.
3 . The computer-implemented method of claim 1 , further comprising: after executing the input block, deploying the output block into the first computer system.
4 . The computer-implemented method of claim 1 , wherein the splitting of the artificial intelligence model is performed based on available resources in the first computer system.
5 . The computer-implemented method of claim 4 , wherein the splitting of the artificial intelligence model is dynamically performed or performed using one of predefined splitting options which are associated with a respective amount of resources.
6 . The computer-implemented method of claim 1 , wherein the execution of the artificial intelligence model comprises execution of a succession of processing steps, wherein splitting the artificial intelligence model is performed such that the input block is configured to perform a first number of successive processing steps, the intermediate block is configured to perform a second number of successive processing steps that follow the first number of successive processing steps of the input block, and the output block is configured to perform a third number of last successive processing steps, wherein a sum of the first number of successive processing steps, the second number of successive processing steps, and the third number of last successive processing steps is a total number of processing steps in the artificial intelligence model.
7 . The computer-implemented method of claim 6 , wherein the first number of successive processing steps is smaller than the second number of successive processing steps by a first delta value, wherein the third number of last successive processing steps is smaller than the second number of successive processing steps by a second delta value.
8 . The computer-implemented method of claim 7 , further comprising determining the first and second delta values based on available resources in the first computer system.
9 . The computer-implemented method of claim 1 , wherein the first encoding protocol is the second encoding protocol.
10 . The computer-implemented method of claim 1 , wherein the first encoding protocol is different from the second encoding protocol.
11 . The computer-implemented method of claim 1 , wherein the first encoding protocol is selected from a group consisting of compression and encryption, and wherein the second encoding protocol is selected from a group consisting of compression and encryption.
12 . The computer-implemented method of claim 1 , wherein the execution of the artificial intelligence model is an inference of the artificial intelligence model which is already trained.
13 . The computer-implemented method of claim 1 , further comprising, for each further received input, training the artificial intelligence model, wherein the first computer system is further configured to compute in each iteration a loss function and to send a result to the second computer system, wherein the result is used by the first and second computer systems to update learnable parameters of the artificial intelligence model, wherein an iteration is performed until the loss function fulfils a convergence criterion.
14 . The computer-implemented method of claim 1 , wherein the first computer system has an amount of processing resources which is smaller than the processing resources of the second computer system.
15 . The computer-implemented method of claim 1 , wherein the first computer system is selected from a group consisting of an edge device and an internet of things (IoT) device.
16 . The computer-implemented method of claim 1 , wherein the second computer system is provided as a service in a cloud environment.
17 . The computer-implemented method of claim 1 , wherein the artificial intelligence model is a foundation model, and wherein the artificial intelligence model is a deep neural network where the input block represents first network layers, the intermediate block represents middle network layers, and the output block represents last network layers.
18 . The computer-implemented method of claim 1 , further comprising splitting the artificial intelligence model by a management server, and deploying by the management server the input block, the output block, and the intermediate block in the first and second computer systems.
19 . A computer program product comprising:
a non-transitory computer-readable storage media having computer-readable program code embodied therewith, the computer-readable program code configured to cause one or more processors to:
receive an input for execution of an artificial intelligence model, wherein the artificial intelligence model is split into an input block, an intermediate block, and an output block, such that the input block receives specific input and provides an intermediate output, the intermediate block receives as input the intermediate output and provides another intermediate output, and the output block receives as input the another intermediate output and provides specific output;
execute the input block by a first computer system using the input, producing a first intermediate output;
encode by the first computer system the first intermediate output using a first encoding protocol to produce an encoded first intermediate output;
send the encoded first intermediate output to a second computer system to allow the second computer system to:
decode the encoded first intermediate output using the first encoding protocol, and execute the intermediate block using as input the first intermediate output to produce a second intermediate output,
encode the second intermediate output using a second encoding protocol to produce an encoded second intermediate output, and send the encoded second intermediate output to the first computer system; and
in response to receiving the encoded second intermediate output, decode by the first computer system the encoded second intermediate output using the second encoding protocol, and execute the output block at the first computer system using as input the second intermediate output, producing a result output of a task performed by the artificial intelligence model as a whole by way of the input block, intermediate block, and output block.
20 . A system for executing an artificial intelligence model, comprising: one or more computer readable storage media storing program instructions and one or more processors which, in response to executing the program instructions, are configured to:
receive an input for execution of the artificial intelligence model, wherein the artificial intelligence model is split into an input block, an intermediate block, and an output block, such that the input block receives a specific input and provides an intermediate output, the intermediate block receives as input the intermediate output and provides another intermediate output, and the output block receives as input the another intermediate output and provides a specific output;
execute the input block using the input received for execution of the artificial intelligence model, producing a first intermediate output;
encode the first intermediate output using a first encoding protocol to produce an encoded first intermediate output;
send the encoded first intermediate output to a second computer system to allow the second computer system to perform at least:
decoding the encoded first intermediate output using the first encoding protocol;
executing the intermediate block as input the first intermediate output, resulting in a second intermediate output;
encoding the second intermediate output using a second encoding protocol to produce an encoded second intermediate output;
sending the encoded second intermediate output to the computer system; and
in response to receiving the encoded second intermediate output, decode the encoded second intermediate output using the second encoding protocol, and execute the output block using as input the second intermediate output, producing a result output of a task performed by the artificial intelligence model as a whole by way of the input block, intermediate block, and output block.