Optimization tool for agentic flows

1 . A method comprising:

obtaining an agentic workflow from an artificial intelligence (AI) agent execution framework;

generating, by a large language model (LLM), an execution graph for the agentic workflow, the execution graph comprising a plurality of nodes corresponding to a plurality of tasks;

partitioning the plurality of tasks by complexity to generate a set of first complexity level tasks and a set of second complexity level tasks;

executing a first set of candidate models to perform the set of first complexity level tasks to assign a first candidate model to a first task of the set of first complexity level tasks;

executing a second set of candidate models to perform the set of second complexity level tasks to assign a second candidate model to a second task of the set of second complexity level tasks;

assigning, by the LLM, a first node corresponding to the first task to the first candidate model, and a second node corresponding to the second task to the second candidate model in the execution graph, to generate a modified execution graph;

deploying an updated agentic workflow generated from the modified execution graph to the AI agent execution framework; and

traversing, by the AI agent execution framework, the modified execution graph, comprising:

causing, by the AI agent execution framework, the first candidate model assigned to the first node to execute the first task,

selecting, by the AI agent execution framework, a subsequent task, based on a result obtained by executing the first task by the first candidate model, and

causing, by the AI agent execution framework, a subsequent candidate model to execute the subsequent task, wherein the subsequent candidate model is assigned to a subsequent node corresponding to the subsequent task.

2 . The method of claim 1 , further comprising:

executing the first set of candidate models by performing operations comprising:

selecting the first candidate model from the first set of candidate models,

selecting a first test dataset corresponding to the first candidate model, and

processing, by the first candidate model, a first task of the set of first complexity level tasks, using the first test dataset, to obtain a set of performance metrics corresponding to the set of first complexity level tasks, wherein the set of performance metrics corresponds to the first candidate model.

3 . The method of claim 1 , further comprising:

executing, by each candidate model of the first set of candidate models, the set of first complexity level tasks to obtain a plurality of sets of performance metrics, wherein each set of performance metrics of the plurality of sets of performance metrics is generated by a corresponding candidate model of the first set of candidate models.

4 . The method of claim 1 , further comprising:

assigning the first candidate model to the first task of the set of first complexity level tasks by performing operations comprising:

obtaining a plurality of sets of performance metrics corresponding to the first set of candidate models,

selecting, for the first task, corresponding performance metrics from the plurality of sets of performance metrics,

applying an optimization function to the corresponding performance metrics to obtain corresponding optimization function values,

selecting a performance metric from the corresponding performance metrics based on a corresponding optimization function value satisfying a minimum accuracy threshold,

obtaining a candidate model from the first set of candidate models that generated the performance metric, and

assigning the candidate model as the first candidate model.

5 . The method of claim 1 , further comprising:

executing the second set of candidate models by performing operations comprising:

selecting the second candidate model from the second set of candidate models,

selecting a second test dataset corresponding to the second candidate model, and

processing, by the second candidate model, each task of the set of second complexity level tasks, to obtain a set of performance metrics corresponding to the set of second complexity level tasks, wherein the set of performance metrics corresponds to the second candidate model.

6 . The method of claim 1 , further comprising:

executing, by each candidate model of the second set of candidate models, the set of second complexity level tasks to obtain a plurality of sets of performance metrics, wherein each set of performance metrics of the plurality of sets of performance metrics is generated by a corresponding candidate model of the second set of candidate models.

7 . The method of claim 1 , further comprising:

assigning the second candidate model to the second task of the set of second complexity level tasks by performing operations comprising:

obtaining a plurality of sets of performance metrics corresponding to the second set of candidate models,

selecting, for the second task, corresponding performance metrics from the plurality of sets of performance metrics,

applying an optimization function to the corresponding performance metrics to obtain corresponding optimization function values,

selecting a performance metric from the corresponding performance metrics based on a corresponding optimization function value satisfying a minimum accuracy threshold,

obtaining a candidate model from the second set of candidate models that generated the performance metric, and

assigning the candidate model as the second candidate model.

8 . The method of claim 1 , wherein the first candidate model executes a tool to perform at least a portion of the first task.

9 . A system, comprising:

at least one computer processor;

an artificial intelligence (AI) agent execution framework, executing on the at least one computer processor; and

an optimization tool, executing on the at least one computer processor, and configured for:

obtaining an agentic workflow from the AI agent execution framework,

generating, by a large language model (LLM) of the optimization tool, an execution graph for the agentic workflow, the execution graph comprising a plurality of nodes corresponding to a plurality of tasks,

partitioning the plurality of tasks by complexity to generate a set of first complexity level tasks and a set of second complexity level tasks,

executing a first set of candidate models to perform the set of first complexity level tasks, to assign a first candidate model to a first task of the set of first complexity level tasks,

executing a second set of candidate models to perform the set of second complexity level tasks to assign a second candidate model to a second task of the set of second complexity level tasks,

assigning, by the LLM of the optimization tool, a first node corresponding to the first task to the first candidate model, and a second node corresponding to the second task to the second candidate model in the execution graph, to generate a modified execution graph,

deploying an updated agentic workflow generated from the modified execution graph to the AI agent execution framework,

wherein the AI agent execution framework performs operations comprising:

traversing, by the AI agent execution framework, the modified execution graph, comprising:

causing the first candidate model assigned to the first node to execute the first task,

selecting a subsequent task, based on a result obtained by executing the first task by the first candidate model, and

causing a subsequent candidate model to execute the subsequent task, wherein the subsequent candidate model is assigned to a subsequent node corresponding to the subsequent task.

10 . The system of claim 9 , further configured for:

executing the first set of candidate models by performing operations comprising:

selecting the first candidate model from the first set of candidate models,

selecting a first test dataset corresponding to the first candidate model, and

processing, by the first candidate model, each task of the set of first complexity level tasks, to obtain a set of performance metrics corresponding to the set of first complexity level tasks, wherein the set of performance metrics corresponds to the first candidate model.

11 . The system of claim 9 , further configured for:

executing by each candidate model of the first set of candidate models, the set of first complexity level tasks to obtain a plurality of sets of performance metrics, wherein each set of performance metrics of the plurality of sets of performance metrics is generated by a corresponding candidate model of the first set of candidate models.

12 . The system of claim 9 , further configured for:

assigning the first candidate model to the first task of the set of first complexity level tasks by performing operations comprising:

obtaining a plurality of sets of performance metrics corresponding to the first set of candidate models,

selecting, for the first task, corresponding performance metrics from the plurality of sets of performance metrics,

applying an optimization function to the corresponding performance metrics to obtain corresponding optimization function values,

selecting a performance metric from the corresponding performance metrics based on a corresponding optimization function value satisfying a minimum accuracy threshold,

obtaining a candidate model from the first set of candidate models that generated the performance metric, and

assigning the candidate model as the first candidate model.

13 . The system of claim 9 , further configured for:

executing the second set of candidate models by performing operations comprising:

selecting the second candidate model from the second set of candidate models,

selecting a second test dataset corresponding to the second candidate model, and

processing, by the second candidate model, each task of the set of second complexity level tasks, to obtain a set of performance metrics corresponding to the set of second complexity level tasks, wherein the set of performance metrics corresponds to the second candidate model.

14 . The system of claim 9 , further configured for:

executing, by each candidate model of the second set of candidate models, the set of second complexity level tasks to obtain a plurality of sets of performance metrics, wherein each set of performance metrics of the plurality of sets of performance metrics is generated by a corresponding candidate model of the second set of candidate models.

15 . The system of claim 9 , further configured for:

assigning the second candidate model to the second task of the set of second complexity level tasks by performing operations comprising:

obtaining a plurality of sets of performance metrics corresponding to the second set of candidate models,

selecting, for the second task, corresponding performance metrics from the plurality of sets of performance metrics,

applying an optimization function to the corresponding performance metrics to obtain corresponding optimization function values,

selecting a performance metric from the corresponding performance metrics based on a corresponding optimization function value satisfying a minimum accuracy threshold,

obtaining a candidate model from the second set of candidate models that generated the performance metric, and

assigning the candidate model as the second candidate model.

16 . A method, comprising:

generating, by an LLM, an execution graph of an agentic workflow, the execution graph comprising a plurality of nodes corresponding to a plurality of tasks;

partitioning the plurality of tasks by complexity to generate a set of first complexity level tasks and a set of second complexity level tasks;

executing by each candidate model of a first set of candidate models, the set of first complexity level tasks to obtain a first plurality of sets of performance metrics, wherein each set of performance metrics of the first plurality of sets of performance metrics is generated by a corresponding candidate model of the first set of candidate models;

selecting, for a first task of the first complexity level tasks, a first plurality of corresponding performance metrics from the first plurality of sets of performance metrics;

applying a first optimization function to the corresponding performance metrics to obtain a first plurality of corresponding optimization function values;

selecting a first performance metric from the corresponding performance metrics based on a first corresponding optimization function value satisfying a minimum accuracy threshold;

assigning a first candidate model from the first set of candidate models that generated the first performance metric;

executing by each candidate model of a second set of candidate models, the set of second complexity level tasks to obtain a second plurality of sets of performance metrics, wherein each set of performance metrics of the second plurality of sets of performance metrics is generated by the corresponding candidate model of the second set of candidate models;

selecting, for a second task of the set of second complexity level tasks, a second plurality of corresponding performance metrics from the second plurality of sets of performance metrics;

applying a second optimization function to the second plurality of corresponding performance metrics to obtain a second plurality of corresponding optimization function values;

selecting a second performance metric from the second plurality of corresponding performance metrics based on a second corresponding optimization function value satisfying the minimum accuracy threshold;

assigning a second candidate model from the second set of candidate models that generated the second performance metric;

assigning, by the LLM, a first node corresponding to the first task to the first candidate model, and a second node corresponding to the second task to the second candidate model in the execution graph, to generate a modified execution graph; and

deploying an updated agentic workflow generated from the modified execution graph to an AI agent execution framework,

wherein the AI agent execution framework performs operations comprising:

traversing, by the AI agent execution framework, the modified execution graph, comprising:

causing, by the AI agent execution framework, the first candidate model assigned to the first node to execute the first task,

selecting, by the AI agent execution framework, a subsequent task, based on a result obtained by executing the first task by the first candidate model, and

causing, by the AI agent execution framework, a subsequent candidate model to execute the subsequent task, wherein the subsequent candidate model is assigned to a subsequent node corresponding to the subsequent task.