Machine learning model management and software development integration
In examples, a declaration of an ML model is identified within source code of a software project. As a result, a model wrapper may be generated for the ML model and used when compiling and/or executing the software code. Further, a representative object may be generated to enable management of the ML model during the software development process. As an example, model attributes associated with the ML model may be identified from the software code and used to manage the ML model accordingly. In examples, a runtime library associated with the ML model may be automatically included in the software project and/or training of the ML model may be automatically initiated. In some instances, a placeholder ML model or a partially trained or intermediate ML model may be used when building and executing the software project while the ML model is still being trained, thereby enabling continued software development.
1 . A system comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations, the set of operations comprising:
identifying a declaration of a machine learning (ML) model within software code of a software project;
in response to identifying the declaration of the ML model within the software code:
initiating training of the ML model using a set of attributes associated with the ML model, wherein the set of attributes is automatically determined based on at least one of software code content, a set of preferences, or received user input; and
generating a model wrapper for the ML model, wherein the model wrapper represents a set of attributes associated at least one of a type of data input or a type of data output for the ML model;
building the software project based on the model wrapper for the ML model;
obtaining an intermediate model for the ML model that is incorporated into the model wrapper for a first execution of the software project, wherein the intermediate model is partially trained as a result of initiating the training of the ML model, wherein the partially trained intermediate model is used while the ML model is still being trained, thereby enabling continued software development; and
outputting the built software project for execution.
2 . The system of claim 1 , wherein the set of operations further comprises:
obtaining a completed model for the ML model that is incorporated into the model wrapper for a second execution or the software project.
3 . The system of claim 1 , wherein:
the set of attributes includes an indication of local training; and
initiating training of the ML model comprises training the ML model based on training data indicated by the software code.
4 . The system of claim 1 , wherein:
the set of attributes includes an indication of remote training; and
initiating training of the ML model comprises providing, to a machine learning service, an indication of the ML model and training data for training the ML model.
5 . The system of claim 1 , wherein building the software project includes generating, based on the model wrapper, an application programming interface call to a machine learning service associated with the ML model.
6 . The system of claim 1 , wherein the set of operations further comprises updating a user interface of a development application displaying at least a part of the software code to include a model entry for the ML model, wherein the model entry includes an indication of at least one of the set of attributes associated with the ML model.
7 . The system of claim 1 , wherein the set of attributes is determined based on one or more of the software code, a user-configured preference, or a default preference.
8 . The system of claim 1 , wherein identifying the declaration of the ML model includes at least one of pattern matching, semantic matching, contextual inference, receiving a user indication of the declaration of the ML model, or identifying a declaration that conforms to a predefined format.
9 . A method, comprising:
identifying a declaration of a machine learning (ML) model within software code of a software project;
in response to identifying the declaration of the ML model within the software code:
providing, to a machine learning service, an indication of the ML model and training data for training the ML model, wherein the software code includes an indication of the training data with which to train the ML model; and
obtaining, from the machine learning service, an intermediate model for the ML model, wherein the intermediate model is partially trained as a result of initiating the training of the ML model, wherein the partially trained intermediate model is used while the ML model is still being trained, thereby enabling continued software development;
building the software project based on the intermediate model, wherein the software project comprises a model wrapper associated with the intermediate model that represents a set of attributes associated with at least one of a type of data input or a type of data output for the intermediate model; and
outputting the built software project for deployment or execution.
10 . The method of claim 9 , further comprising generating the model wrapper for the software project in response to identifying the declaration of the ML model within the software code that incorporates the intermediate model from the machine learning service.
11 . The method of claim 9 , wherein outputting the built software includes invoking a debugger interface to perform executable debugging of the built software project.
12 . The method of claim 9 , further comprising:
obtaining, from the machine learning service, and indication of a training status for the ML model; and
updating a user interface of a development application displaying at least a part of the software code to include a model entry for the ML model, wherein the model entry includes an indication of the training status for the ML model.
13 . The method of claim 9 , further comprising:
obtaining, from the machine learning service, a second model comprising a second intermediate model or a fully-trained model for the ML model; and
building the software project based on the second model.
14 . The method of claim 9 , wherein building the software project comprises generating a software executable that includes, as a result of identifying the declaration of the ML model, a runtime library associated with the ML model.
15 . A method, comprising:
identifying a declaration of a machine learning (ML) model within software code of a software project;
in response to identifying the declaration of the ML model within the software code:
initiating automatic training of the ML model using a set of attributes associated with the ML model, wherein the set of attributes is automatically determined based on at least one of software code content, a set of preferences, or received user input; and
generating a model wrapper for the ML model, wherein the model wrapper represents a set of attributes associated at least one of a type of data input or a type of data output for the ML model;
obtaining an intermediate model for the ML model that is incorporated into the model wrapper for a first execution of the software project, wherein the intermediate model is partially trained as a result of initiating the training of the ML model, wherein the partially trained intermediate model is used while the ML model is still being trained, thereby enabling continued software development; and
executing the software project based on the model wrapper for the ML model.
16 . The method of claim 15 , further comprising:
obtaining a completed model for the ML model that is incorporated into the model wrapper for a second execution of the software project.
17 . The method of claim 15 , wherein:
the set of attributes includes an indication of remote training; and
initiating training of the ML model comprises providing, to a machine learning service, an indication of the ML model and training data for training the ML model.
18 . The method of claim 15 , wherein executing the software project includes generating, based on the model wrapper, an application programming interface call to a machine learning service associated with the ML model.
19 . The method of claim 15 , further comprising updating a user interface of a development application displaying at least a part of the software code to include a model entry for the ML model, wherein the model entry includes an indication of at least one of the set of attributes associated with the ML model.
20 . The method of claim 15 , wherein the set of attributes is determined based on one or more of the software code, a user-configured preference, or a default preference.