Machine learning model retraining pipeline for robotic process automation
A machine learning (ML) model retraining pipeline for robotic process automation (RPA) is disclosed. When an ML model is deployed in a production or development environment, RPA robots send requests to the ML model when executing their workflows. When a confidence level of the ML model falls below a certain confidence, training data is collected, potentially from a large number of computing systems. The ML model is then trained using at least in part the collected training data, and a new version of the ML model is deployed.
1. A computer-implemented method for implementing a machine learning (ML) model retraining pipeline for robotic process automation (RPA), comprising:
calling a machine learning (ML) model, by an RPA robot, while executing an activity of an RPA workflow that uses the ML model;
receiving a result from the execution of the ML model, by the RPA robot;
completing execution of the activity of the ROA RPA workflow using the result from the execution of the ML model, by the RPA robot;
determining whether one or more trigger conditions are met for labeling of data for the ML model, by the RPA robot; and
responsive to the one or more trigger conditions being met:
prompting a user to provide labeled data for training or retraining the ML model and sending the labeled data to a server for training or retraining of the ML model, by the RPA robot, or adding information pertaining to the result from the ML model to a queue for subsequent labeling, by the RPA robot, wherein
the RPA workflow comprises a plurality of activities that comprise the one or more activities, and
the activities are a set of steps developed in the workflow.
2. The computer-implemented method of claim 1 , wherein the one or more trigger conditions comprise ML model performance falling below a confidence threshold, ML model results deviating from a statistical distribution, ML model results that fall outside of a previously observed range, or any combination thereof.
3. The computer-implemented method of claim 1 , wherein the one or more trigger conditions comprise receipt of an initiation command for collection of the labeled data.
4. The computer-implemented method of claim 1 , further comprising: receiving the labeled data, by the server; and storing the labeled data in a database, by the server.
5. The computer-implemented method of claim 1 , further comprising: responsive to one or more training criteria being met, training the ML model, by the server, using the labeled data to produce a new version of the ML model; and deploying the new version of the ML model, by the server.
6. The computer-implemented method of claim 5 , wherein the training of the ML model additionally comprises using previous training data that was used to train a version of the ML model trained prior to the new version of the ML model.
7. The computer-implemented method of claim 5 , wherein the one or more training criteria comprise a predetermined time period elapsing, a predetermined amount of training data being received, a predetermined number of training data samples being received, the data falling outside of a distribution of data, or any combination thereof.
8. The computer-implemented method of claim 5 , wherein the new version of the ML model replaces a previous version of the ML model.
9. The computer-implemented method of claim 5 , further comprising: modifying a workflow of the RPA robot to call the new version of the ML model.
10. The computer-implemented method of claim 5 , further comprising: modifying a workflow of the RPA robot to call multiple versions of the ML model, the multiple versions comprising the new version of the ML model.
11. The computer-implemented method of claim 10 , further comprising: calling the new version of the ML model, by the RPA robot; and responsive to the one or more trigger conditions not being met, calling a previous version of the ML model and selecting a best result from the current version of the ML model and the previous version of the ML model, by the RPA robot.
12. The computer-implemented method of claim 10 , further comprising: calling the new version of the ML model and at least one previous version of the ML model, by the RPA robot; and selecting a best result from the current version of the ML model and the at least one previous version of the ML model, by the RPA robot.
13. The computer-implemented method of claim 1 , wherein collection of the labeled data and training of the new version of the ML model is uniform for a plurality of ML models called by a plurality of RPA robots.
14. A computer-implemented method for implementing a machine learning (ML) model retraining pipeline for robotic process automation (RPA), comprising: determining whether one or more trigger conditions are met for labeling of data for an ML model, by an RPA robot, the RPA robot configured to call a machine learning (ML) model, by an RPA robot, responsive to the RPA robot executing an activity of an RPA workflow that uses the ML model; and
responsive to the one or more trigger conditions being met:
prompting a user to provide labeled data for training or retraining the ML model and sending the labeled data to a server for training or retraining of the ML model, by the RPA robot, or adding information pertaining to the result from the ML model to a queue for subsequent labeling, by the RPA robot, wherein
the RPA workflow comprises a plurality of activities that comprise the one or more activities, and
the activities are a set of steps developed in the workflow.
15. The computer-implemented method of claim 14 , wherein the one or more trigger conditions comprise ML model performance falling below a confidence threshold, ML model results deviating from a statistical distribution, ML model results that fall outside of a previously observed range, or any combination thereof.
16. The computer-implemented method of claim 14 , further comprising: responsive to one or more training criteria being met, training the ML model, by the server, using the labeled data to produce a new version of the ML model; and deploying the new version of the ML model, by the server.
17. The computer-implemented method of claim 16 , wherein the one or more training criteria comprise a predetermined time period elapsing, a predetermined amount of training data being received, a predetermined number of training data samples being received, the data falling outside of a distribution of data, or any combination thereof.
18. The computer-implemented method of claim 16 , further comprising: modifying a workflow of the RPA robot to call multiple versions of the ML model, the multiple versions comprising the new version of the ML model.
19. A computer-implemented method for implementing a machine learning (ML) model retraining pipeline for robotic process automation (RPA), comprising:
calling a machine learning (ML) model, by an RPA robot, while executing an activity of an RPA workflow that uses the ML model;
receiving a result from the execution of the ML model, by the RPA robot;
completing execution of the activity of the RPA workflow using the result from the execution of the ML model, by the RPA robot;
determining whether one or more trigger conditions are met for labeling of data for the ML model, by the RPA robot; responsive to the one or more trigger conditions being met:
prompting a user to provide labeled data for training or retraining the ML model and sending the labeled data to a server for training or retraining of the ML model, by the RPA robot, or adding information pertaining to the result from the ML model to a queue for subsequent labeling, by the RPA robot;
responsive to one or more training criteria being met, training the ML model, by the server, using the labeled data to produce a new version of the ML model; and
deploying the new version of the ML model, by the server, wherein
the RPA workflow comprises a plurality of activities that comprise the one or more activities, and
the activities are a set of steps developed in the workflow.
20. The computer-implemented method of claim 19 , wherein the one or more trigger conditions comprise ML model performance falling below a confidence threshold, ML model results deviating from a statistical distribution, ML model results that fall outside of a previously observed range, or any combination thereof, and the one or more training criteria comprise a predetermined time period elapsing, a predetermined amount of training data being received, a predetermined number of training data samples being received, the data falling outside of a distribution of data, or any combination thereof.