Method and system for an end-to-end artificial intelligence workflow
In general, certain embodiments of the present disclosure provide methods and systems for enabling a reproducible processing of machine learning models and scalable deployment on a distributed network. The method comprises building a machine learning model; training the machine learning model to produce a plurality of versions of the machine learning model; tracking the plurality of versions of the machine learning model to produce a change facilitator tool; sharing the change facilitator tool to one or more devices such that each device can reproduce the plurality of versions of the machine learning model; and generating a deployable version of the machine learning model through repeated training.
1. A system for enabling a reproducible processing of machine learning models and scalable deployment on a distributed network, comprising:
one or more processors;
memory; and
one or more programs stored in the memory, the one or more programs comprising instructions for:
building a machine learning model;
training the machine learning model to produce a plurality of versions of the machine learning model;
tracking the plurality of versions of the machine learning model to produce a change facilitator tool;
sharing the change facilitator tool to one or more devices such that each device can reproduce the plurality of versions of the machine learning model; and
generating a deployable version of the machine learning model through repeated training.
2. The system of claim 1 , wherein tracking the plurality of versions includes tracking 5 components: source code, software environments, files, configuration parameters, and performance metrics.
3. The system of claim 2 , wherein the 5 components are tracked at the same time.
4. The system of claim 3 , wherein each version in the plurality of versions is a snapshot of all 5 components being tracked.
5. The system of claim 4 , wherein each snapshot allows for complete reproducibility across any machine.
6. The system of claim 4 , wherein the plurality of versions allows for running multiple tasks and pulling from multiple different snapshots in order to widen the search space to improve performance metrics to create improved snapshots.
7. The system of claim 1 , wherein the deployable version corresponds to the version with the best performing metrics.
8. A method for enabling a reproducible processing of machine learning models and scalable deployment on a distributed network comprising:
building a machine learning model;
training the machine learning model to produce a plurality of versions of the machine learning model;
tracking the plurality of versions of the machine learning model to produce a change facilitator tool;
sharing the change facilitator tool to one or more devices such that each device can reproduce the plurality of versions of the machine learning model; and
generating a deployable version of the machine learning model through repeated training.
9. The method of claim 8 , wherein tracking the plurality of versions includes tracking 5 components: source code, software environments, files, configuration parameters, and performance metrics.
10. The method of claim 9 , wherein the 5 components are tracked at the same time.
11. The method of claim 10 , wherein each version in the plurality of versions is a snapshot of all 5 components being tracked.
12. The method of claim 11 , wherein each snapshot allows for complete reproducibility across any machine.
13. The method of claim 11 , wherein the plurality of versions allows for running multiple tasks and pulling from multiple different snapshots in order to widen the search space to improve performance metrics to create improved snapshots.
14. The method of claim 8 , wherein the deployable version corresponds to the version with the best performing metrics.
15. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:
building a machine learning model;
training the machine learning model to produce a plurality of versions of the machine learning model;
tracking the plurality of versions of the machine learning model to produce a change facilitator tool;
sharing the change facilitator tool to one or more devices such that each device can reproduce the plurality of versions of the machine learning model; and
generating a deployable version of the machine learning model through repeated training.
16. The non-transitory computer readable medium of claim 15 , wherein tracking the plurality of versions includes tracking 5 components: source code, software environments, files, configuration parameters, and performance metrics.
17. The non-transitory computer readable medium of claim 16 , wherein the 5 components are tracked at the same time.
18. The non-transitory computer readable medium of claim 17 , wherein each version in the plurality of versions is a snapshot of all 5 components being tracked.
19. The non-transitory computer readable medium of claim 18 , wherein each snapshot allows for complete reproducibility across any machine.
20. The non-transitory computer readable medium of claim 18 , wherein the plurality of versions allows for running multiple tasks and pulling from multiple different snapshots in order to widen the search space to improve performance metrics to create improved snapshots.