Augmentation of machine learning pipeline corpus for synthesizing new machine learning pipelines
According to an aspect of an embodiment, operations may include receiving an ML project stored in an ML corpus database. The operations may further include mutating a first ML pipeline, of a set of first ML pipelines associated with the received ML project, to determine a set of second ML pipelines. The mutation of the first ML pipeline may correspond to a substitution of a first ML model associated with the first ML pipeline with a second ML model associated with one of the set of predefined ML pipelines. The operations may further include selecting one or more ML pipelines from the set of second ML pipelines based on a performance score associated with each of the determined set of ML pipelines. The operations may further include augmenting the ML corpus database to include the selected one or more ML pipelines and the set of first ML pipeline.
1 . A method, executed by a processor, comprising:
receiving a machine learning (ML) project from a plurality of ML projects stored in an ML corpus database,
wherein each of the plurality of ML projects includes a dataset and
a set of ML pipelines applicable on the dataset;
mutating a first ML pipeline, of a set of first ML pipelines associated with the received ML project, to determine a set of second ML pipelines, based on a set of predefined ML pipelines,
wherein the mutation of the first ML pipeline corresponds to a substitution of a first ML model, associated with the first ML pipeline, with
a second ML model associated with one of the set of predefined ML pipelines;
selecting one or more ML pipelines from the determined set of second ML pipelines based on a performance score associated with each of the determined set of second ML pipelines;
augmenting the ML corpus database to include the selected one or more ML pipelines and the set of first ML pipelines associated with the received ML project;
constructing an abstract syntax tree (AST) associated with the first ML pipeline;
determining a last application programming interface (API) call associated with a predict function in the first ML pipeline based on the constructed AST;
assigning the determined last API call as a target line; and
identifying one or more statements associated with the first ML model based on the assigned target line.
2 . The method according to claim 1 , further comprising:
extracting a set of meta-features from the dataset associated with each of the plurality of ML projects stored in the augmented ML corpus database;
extracting a set of ML pipeline components from the set of ML pipelines associated with each of the plurality of ML projects stored in the augmented ML corpus database; and
training a meta-learning model based on the extracted set of meta- features and the extracted set of ML pipeline components.
3 . The method according to claim 1 , wherein the performance score associated with each of the determined set of second ML pipelines corresponds to at least one of: an F1-score or an R2-score associated with the corresponding ML pipeline.
4 . The method according to claim 1 , wherein the selected one or more ML pipelines from the determined set of second ML pipelines includes one of:
an ML pipeline, from the determined set of second ML pipelines, associated with a maximum performance score, a first group of ML pipelines, from the determined set of second ML pipelines, that correspond to a performance score above a threshold, or a second group of ML pipelines, from the determined set of second ML pipelines, that correspond to a predefined number of top ML pipelines based on the performance score.
5 . The method according to claim 1 , further comprising:
identifying a code snippet of the first ML model associated with the first ML pipeline;
determining one or more input parameters associated with the identified code snippet;
selecting the second ML model from a set of predefined models associated with the set of predefined ML pipelines; and
instantiating the selected second ML model based on the substitution of the first ML model with the selected second ML model in the first ML pipeline.
6 . The method according to claim 5 , wherein the determined one or more input parameters associated with the identified code snippet comprises at least one of: a training dataset, a test dataset, and a set of hyper-parameters associated with the first ML model.
7 . The method according to claim 5 , further comprising:
selecting a predefined template associated with the second ML model, the selected predefined template is annotated with the one or more input parameters of the identified code snippet of the first ML model;
constructing a code snippet of the second ML model based on a parameterization of one or more function calls in the selected predefined template using the annotated one or more input parameters; and
replacing the identified code snippet of the first ML model with the constructed code snippet of the second ML model to instantiate the second ML model.
8 . The method according to claim 1 , wherein the one or more statements associated with the first ML model are identified based on an application of a backward program slicing from the assigned target line until a model declaration associated with the first ML model is reached.
9 . The method according to claim 1 , further comprising storing a line number of each of the one or more statements associated with the first ML model,
wherein the one or more statements correspond to at least one of: a model definition, a fit function call, or a predict function call.
10 . The method according to claim 1 , further comprising:
retrieving the assigned target line from the first ML pipeline;
adding the retrieved target line into a queue including a set of statements associated with the first ML pipeline;
popping a first statement from the queue;
controlling an execution of a first set of operations to obtain a model slice associated with the first ML model from the first ML pipeline, wherein the first set of operations includes:
extracting one or more variables and objects from the first statement,
identifying a set of second statements that occur prior to the first statement in the first ML pipeline and include at least one of the extracted one or more variables and objects,
determining whether a third statement of the identified set of second statements occurs prior to a model definition associated with the first ML model,
adding the third statement to the queue based on the determination that the third statement occurs prior to the model definition,
adding the first statement to the model slice, and
popping a fourth statement from the queue as the first statement, based on a determination that the queue is not empty; and
obtaining the model slice to identify a code snippet of the first ML model associated with the first ML pipeline, based on an iterative execution of the first set of operations.
11 . One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations comprising:
receiving a machine learning (ML) project from a plurality of ML projects stored in an ML corpus database,
wherein each of the plurality of ML projects includes a dataset and a set of ML pipelines applicable on the dataset;
mutating a first ML pipeline, of a set of first ML pipelines associated with the received ML project, to determine a set of second ML pipelines, based on a set of predefined ML pipelines,
wherein the mutation of the first ML pipeline corresponds to a substitution of a first ML model, associated with the first ML pipeline, with a second ML model associated with one of the set of predefined ML pipelines;
selecting one or more ML pipelines from the determined set of second ML pipelines based on a performance score associated with each of the determined set of second ML pipelines;
augmenting the ML corpus database to include the selected one or more ML pipelines and the set of first ML pipelines associated with the received ML project;
constructing an abstract syntax tree (AST) associated with the first ML pipeline;
determining a last application programming interface (API) call associated with a predict function in the first ML pipeline based on the constructed AST;
assigning the determined last API call as a target line; and
identifying one or more statements associated with the first ML model based on the assigned target line.
12 . The one or more non-transitory computer-readable storage media according to claim 11 , wherein the operations further comprise:
extracting a set of meta-features from the dataset associated with each of the plurality of ML projects stored in the augmented ML corpus database;
extracting a set of ML pipeline components from the set of ML pipelines associated with each of the plurality of ML projects stored in the augmented ML corpus database; and
training a meta-learning model based on the extracted set of meta-features and the extracted set of ML pipeline components.
13 . The one or more non-transitory computer-readable storage media according to claim 12 , wherein the performance score associated with each of the determined set of second ML pipelines corresponds to at least one of an F1-score or an R2-score associated with the corresponding ML pipeline.
14 . The one or more non-transitory computer-readable storage media according to claim 11 , wherein the selected one or more ML pipelines from the determined set of second ML pipelines includes one of:
an ML pipeline, from the determined set of second ML pipelines, associated with a maximum performance score,
a first group of ML pipelines, from the determined set of second ML pipelines, that correspond to a performance score above a threshold, or
a second group of ML pipelines, from the determined set of second ML pipelines, that correspond to a predefined number of top ML pipelines based on the performance score.
15 . The one or more non-transitory computer-readable storage media according to claim 11 , wherein the operations further comprise:
identifying a code snippet of the first ML model associated with the first ML pipeline;
determining one or more input parameters associated with the identified code snippet;
selecting the second ML model from a set of predefined models associated with the set of predefined ML pipelines; and
instantiating the selected second ML model based on the substitution of the first ML model with the selected second ML model in the first ML pipeline.
16 . The one or more non-transitory computer-readable storage media according to claim 15 , wherein the determined one or more input parameters associated with the identified code snippet comprises at least one of: a training dataset, a test dataset, and a set of hyper-parameters associated with the first ML model.
17 . The one or more non-transitory computer-readable storage media according to claim 15 , wherein the operations further comprise:
selecting a predefined template associated with the second ML model, the selected predefined template is annotated with the one or more input parameters of the identified code snippet of the first ML model;
constructing a code snippet of the second ML model based on a parameterization of one or more function calls in the selected predefined template using the annotated one or more input parameters; and
replacing the identified code snippet of the first ML model with the constructed code snippet of the second ML model to instantiate the second ML model.
18 . An electronic device, comprising:
a memory storing instructions; and
a processor, coupled to the memory, that executes the instructions to perform a process comprising:
receiving a machine learning (ML) project from a plurality of ML projects stored in an ML corpus database,
wherein each of the plurality of ML projects includes a dataset and a set of ML pipelines applicable on the dataset;
mutating a first ML pipeline, of a set of first ML pipelines associated with the received ML project, to determine a set of second ML pipelines, based on a set of predefined ML pipelines,
wherein the mutation of the first ML pipeline corresponds to a substitution of a first ML model, associated with the first ML pipeline, with a second ML model associated with one of the set of predefined ML pipelines;
selecting one or more ML pipelines from the determined set of second ML pipelines based on a performance score associated with each of the determined set of second ML pipelines;
augmenting the ML corpus database to include the selected one or more ML pipelines and the set of first ML pipelines associated with the received ML project;
constructing an abstract syntax tree (AST) associated with the first ML pipeline;
determining a last application programming interface (API) call associated with a predict function in the first ML pipeline based on the constructed AST;
assigning the determined last API call as a target line; and
identifying one or more statements associated with the first ML model based on the assigned target line.