Orchestrator for machine learning pipeline
Provided is a system and method for training and validating models in a machine learning pipeline for failure mode analytics. The machine learning pipeline may include an unsupervised training phase, a validation phase and a supervised training and scoring phase. In one example, the method may include receiving an identification of a machine learning model, executing a machine learning pipeline comprising a plurality of services which train the machine learning model via at least one of an unsupervised learning process and a supervised learning process, the machine learning pipeline being controlled by an orchestration module that triggers ordered execution of the services, and storing the trained machine learning model output from the machine learning pipeline in a database associated with the machine learning pipeline.
1. A computing system comprising:
a storage device configured to store a machine learning model; and
a processor configured to:
establish a first pipeline, via a host platform, which comprises a first plurality of services for unsupervised learning of the machine learning model,
establish a second pipeline, via the host platform, which comprises a second plurality of services for supervised learning of the machine learning model, the second pipeline being independent from the first pipeline,
trigger execution of the first plurality of services via the first pipeline and train the machine learning model via unsupervised learning to generate an initially trained model,
trigger performance of one or more validation tasks for the initially trained model,
store the initially trained model in the storage device in response to performing the one or more validation tasks,
trigger execution of the second plurality of services via the second pipeline and train the initially trained model via supervised learning to generate a further trained model, and
store the further trained model in the storage device.
2. The computing system of claim 1 , wherein the processor is configured to establish services for generating the machine learning model via the first pipeline and establish services for validating the machine learning model generated via the first pipeline based on feedback via a user interface.
3. The computing system of claim 1 , wherein the processor is configured to establish at least one service in common in both the first pipeline and the second pipeline.
4. The computing system of claim 1 , wherein the processor is further configured to generate a validation task, via an orchestrator of the host platform, based on a training result generated from execution of the machine learning model via the first pipeline.
5. The computing system of claim 4 , wherein the processor is further configured to display a user interface via a user device after execution of the first plurality of services via the first pipeline, wherein the user interface comprises a display of the training result and an input mechanism for modifying and validating the training result.
6. The computing system of claim 1 , wherein the processor is configured to instruct the first plurality of services to retrieve a first period of data from a database and score the first period of data via unsupervised learning.
7. The computing system of claim 6 , wherein the processor is configured to instruct the second plurality of services to retrieve a second period of data from the database, which is different from the first period of data, and score the second period of data via supervised learning.
8. The computing system of claim 1 , wherein the processor is configured to control an analytic engine to execute the first plurality of services in sequence, move the analytic engine from the first pipeline to the second pipeline, and control the analytic engine to execute the second plurality of services in sequence.
9. A method comprising:
establishing a first pipeline, via a host platform, which comprises a first plurality of services for unsupervised learning of a machine learning model;
establishing a second pipeline, via the host platform, which comprises a second plurality of services for supervised learning of the machine learning model, the second pipeline being independent from the first pipeline;
triggering execution of the first plurality of services via the first pipeline and train the machine learning model via unsupervised learning to generate an initially trained model;
triggering performance of one or more validation tasks for the initially trained model;
storing the initially trained model in the storage device in response to performing the one or more validation tasks;
triggering execution of the second plurality of services via the second pipeline and train the initially trained model via supervised learning to generate a further trained model; and
storing the further trained model in a storage device.
10. The method of claim 9 , wherein the establishing the first pipeline comprises establishing services for generating the machine learning model and the establishing the second pipeline comprises establishing services for validating the machine learning model generated via the first pipeline based on feedback via a user interface.
11. The method of claim 9 , wherein the establishing the second pipeline comprises establishing at least one service in common in from the first pipeline within the second pipeline.
12. The method of claim 9 , wherein the method further comprises creating a validation task, via an orchestrator of the host platform, based on a training result generated from execution of the machine learning model via the first pipeline.
13. The method of claim 12 , wherein the method further comprises displaying a user interface via a user device after execution of the first plurality of services via the first pipeline, wherein the user interface comprises a display of the training result and an input mechanism for modifying and validating the training result.
14. The method of claim 9 , wherein the triggering execution of the first plurality of services comprises triggering the first plurality of services to retrieve a first period of data from a database and scoring the first period of data via unsupervised learning.
15. The method of claim 14 , wherein the triggering execution of the second plurality of services comprises triggering the second plurality of services to retrieve a second period of data from the database, which is different from the first period of data, and scoring the second period of data via supervised learning.
16. The method of claim 9 , wherein the triggering execution of the first plurality of services comprises controlling an analytic engine to execute the first plurality of services in sequence, and the triggering execution of the second plurality of services comprises moving the analytic engine from the first pipeline to the second pipeline and controlling the analytic engine to execute the second plurality of services in sequence.
17. A computer-readable medium comprising instructions which when executed by a processor cause a computer to perform a method comprising:
establishing a first pipeline, via a host platform, which comprises a first plurality of services for unsupervised learning of a machine learning model;
establishing a second pipeline, via the host platform, which comprises a second plurality of services for supervised learning of the machine learning model, the second pipeline being independent from the first pipeline;
triggering execution of the first plurality of services via the first pipeline and train the machine learning model via unsupervised learning to generate an initially trained model;
triggering performance of one or more validation tasks for the initially trained model;
storing the initially trained model in the storage device in response to performing the one or more validation tasks;
triggering execution of the second plurality of services via the second pipeline and train the initially trained model via supervised learning to generate a further trained model; and
storing the further trained model in a storage device.
18. The computer-readable medium of claim 17 , wherein the establishing the first pipeline comprises establishing services for generating the machine learning model and the establishing the second pipeline comprises establishing services for validating the machine learning model generated via the first pipeline based on feedback via a user interface.
19. The computer-readable medium of claim 17 , wherein the establishing the second pipeline comprises establishing at least one service in common in from the first pipeline within the second pipeline.
20. The computer-readable medium of claim 17 , wherein the triggering execution of the first plurality of services comprises controlling an analytic engine to execute the first plurality of services in sequence, and the triggering execution of the second plurality of services comprises moving the analytic engine from the first pipeline to the second pipeline and controlling the analytic engine to execute the second plurality of services in sequence.