Method and system for an end-to-end artificial intelligence workflow
In general, certain embodiments of the present disclosure provide methods and systems for enabling a reproducible processing of machine learning models and scalable deployment on a distributed network. The method comprises building a machine learning model; training the machine learning model to produce a plurality of versions of the machine learning model; tracking the plurality of versions of the machine learning model to produce a change facilitator tool; sharing the change facilitator tool to one or more devices such that each device can reproduce the plurality of versions of the machine learning model; and generating a deployable version of the machine learning model through repeated training.
1 . A system comprising:
one or more processors;
memory; and
one or more programs stored in the memory, the one or more programs comprising instructions for:
building a machine learning model using a graphical user interface (GUI) and a command line interface (CLI), the GUI and the CLI connecting to an application programming interface (API);
training the machine learning model to produce and track a plurality of snapshots of the machine learning model using the GUI through an adjustment of weights in a repeated process, each snapshot being a self-contained, reproducible version of the machine learning model that includes all relevant components for reproducing the machine learning model, including source code, files, environments, configurations, and performance metrics, wherein each snapshot can be used to fully reproduce the machine learning model in multiple different environments;
producing a change facilitator tool based on the plurality of snapshots of the machine learning model;
sharing the change facilitator tool to one or more devices such that each device can obtain one or more of the plurality of snapshots of the machine learning model; and
facilitating deployment of one or more of the plurality of snapshots of the machine learning model.
2 . The system of claim 1 , wherein tracking the plurality of versions includes tracking one or more of the following components: source code, software environments, files, configuration parameters, and performance metrics.
3 . The system of claim 2 , wherein more than one component is tracked at the same time.
4 . The system of claim 3 , wherein each version in the plurality of versions is a snapshot of the one or more components being tracked.
5 . The system of claim 4 , wherein a snapshot allows for reproducing a version of the machine learning model across a plurality of machines.
6 . The system of claim 1 , wherein the plurality of versions of the machine learning model is stored in persistent storage.
7 . The system of claim 1 , wherein the change facilitator tool is downloaded onto one or more devices.
8 . A method comprising:
building a machine learning model using a graphical user interface (GUI) and a command line interface (CLI), the GUI and the CLI connecting to an application programming interface (API);
training the machine learning model to produce and track a plurality of snapshots of the machine learning model using the GUI through an adjustment of weights in a repeated process, each snapshot being a self-contained, reproducible version of the machine learning model that includes all relevant components for reproducing the machine learning model, including source code, files, environments, configurations, and performance metrics, wherein each snapshot can be used to fully reproduce the machine learning model in multiple different environments;
producing a change facilitator tool based on the plurality of snapshots of the machine learning model;
sharing the change facilitator tool to one or more devices such that each device can obtain one or more of the plurality of snapshots of the machine learning model; and
facilitating deployment of one or more of the plurality of snapshots of the machine learning model.
9 . The method of claim 8 , wherein tracking the plurality of versions includes tracking one or more of the following components: source code, software environments, files, configuration parameters, and performance metrics.
10 . The method of claim 9 , wherein more than one component is tracked at the same time.
11 . The method of claim 10 , wherein each version in the plurality of versions is a snapshot of the one or more components being tracked.
12 . The method of claim 11 , wherein a snapshot allows for reproducing a version of the machine learning model across a plurality of machines.
13 . The method of claim 8 , wherein the plurality of versions of the machine learning model is stored in persistent storage.
14 . The method of claim 8 , wherein the change facilitator tool is downloaded onto one or more devices.
15 . A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:
building a machine learning model using a graphical user interface (GUI) and a command line interface (CLI), the GUI and the CLI connecting to an application programming interface (API);
training the machine learning model to produce and track a plurality of snapshots of the machine learning model using the GUI through an adjustment of weights in a repeated process, each snapshot being a self-contained, reproducible version of the machine learning model that includes all relevant components for reproducing the machine learning model, including source code, files, environments, configurations, and performance metrics, wherein each snapshot can be used to fully reproduce the machine learning model in multiple different environments;
producing a change facilitator tool based on the plurality of snapshots of the machine learning model;
sharing the change facilitator tool to one or more devices such that each device can obtain one or more of the plurality of snapshots of the machine learning model; and
facilitating deployment of one or more of the plurality of snapshots of the machine learning model.
16 . The non-transitory computer readable storage medium of claim 15 , wherein tracking the plurality of versions includes tracking one or more of the following components: source code, software environments, files, configuration parameters, and performance metrics.
17 . The non-transitory computer readable storage medium of claim 16 , wherein more than one component is tracked at the same time.
18 . The non-transitory computer readable storage medium of claim 17 , wherein each version in the plurality of versions is a snapshot of the one or more components being tracked.
19 . The non-transitory computer readable storage medium of claim 18 , wherein a snapshot allows for reproducing a version of the machine learning model across a plurality of machines.
20 . The non-transitory computer readable storage medium of claim 15 , wherein the plurality of versions of the machine learning model is stored in persistent storage.