Method and system for optimizing learning models post-deployment
A method and system for optimizing a learning model post-deployment. Specifically, the disclosed method and system re-optimize—i.e., re-train and/or re-validate—machine learning and/or artificial intelligence algorithms that have already been deployed into a production environment. During post-deployment, the re-optimization process may transpire following the advent of varying model re-adjustment triggers.
1. A method for optimizing learning models post-deployment, comprising:
deploying, to a client device, a first learning model configured with a first adjusted model state;
in response to detecting a model re-adjustment trigger:
selecting a second learning model configured with a second adjusted model state;
making a first determination that the second learning model outperforms the first learning model; and
deploying, based on the first determination and to replace the first learning model,
the second learning model to the client device,
wherein the model re-adjustment trigger comprises one selected from a group consisting of receiving at least a threshold of new records pertinent to and since deploying the first learning model, tracking at least a threshold time interval elapsed since deploying the first learning model, and computing a model performance metric for the first learning model since deploying the first learning model, wherein the model performance metric falls short of a threshold model performance metric.
2. The method of claim 1 , further comprising:
monitoring the second learning model while deployed on the client device;
making a second determination, based on monitoring the second learning model, that the first learning model outperforms the second learning model; and
re-deploying, based on the second determination and to replace the second learning model,
the first learning model to the client device.
3. The method of claim 1 , wherein the first adjusted model state comprises a set of model-specific parameters and hyper-parameters for the first learning model.
4. The method of claim 1 , wherein each new record pertinent to the first learning model comprises a set of feature values representing a sample relevant to a learning model task of the first learning model, and a model prediction for the sample obtained through processing of the set of feature values using the first learning model.
5. The method of claim 4 , wherein each new record pertinent to the first learning model further comprises an actual finding for the sample.
6. The method of claim 1 , further comprising:
prior to deploying the first learning model:
obtaining a default model state;
initializing the first learning model with the default model state; and
adjusting the first learning model using a first record set to obtain the first learning model configured with the first adjusted model state.
7. The method of claim 6 , wherein selecting the second learning model, comprises:
initializing the second learning model with the first adjusted model state;
adjusting the second learning model using a second record set to obtain the second learning model configured with the second adjusted model state;
initializing a third learning model with the default model state;
adjusting the third learning model using a third record set to obtain the third learning model configured with a third adjusted model state;
making a second determination that the second learning model outperforms the third learning model; and
selecting the second learning model based on the second determination.
8. The method of claim 7 , wherein the second record set excludes the first record set,
wherein the third record set comprises the first and second record sets.
9. The method of claim 6 , wherein selecting the second learning model, comprises:
initializing a third learning model with the first adjusted model state;
adjusting the third learning model using a second record set to obtain the third learning model configured with a third adjusted model state;
initializing the second learning model with the default model state;
adjusting the second learning model using a third record set to obtain the second learning model configured with the second adjusted model state;
making a second determination that the second learning model outperforms the third learning model; and
selecting the second learning model based on the second determination.
10. A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to:
deploy, to a client device, a first learning model configured with a first adjusted model state;
in response to detecting a model re-adjustment trigger:
select a second learning model configured with a second adjusted model state;
make a first determination that the second learning model outperforms the first learning model; and
deploy, based on the first determination and to replace the first learning model, the second learning model to the client device,
wherein the model re-adjustment trigger comprises one selected from a group consisting of receiving at least a threshold of new records pertinent to and since deploying the first learning model, tracking at least a threshold time interval elapsed since deploying the first learning model, and computing a model performance metric for the first learning model since deploying the first learning model, wherein the model performance metric falls short of a threshold model performance metric.
11. The non-transitory CRM of claim 10 , comprising computer readable program code,
which when executed by the computer processor, further enables the computer processor to:
monitor the second learning model while deployed on the client device;
make a second determination, based on monitoring the second learning model, that the first learning model outperforms the second learning model; and
re-deploying, based on the second determination and to replace the second learning model,
the first learning model to the client device.
12. The non-transitory CRM of claim 11 , wherein the first adjusted model state comprises a set of model-specific parameters and hyper-parameters for the first learning model.
13. The non-transitory CRM of claim 10 , wherein each new record pertinent to the first learning model comprises a set of feature values representing a sample relevant to a learning model task of the first learning model, and a model prediction for the sample obtained through processing of the set of feature values using the first learning model.
14. The non-transitory CRM of claim 13 , wherein each new record pertinent to the first learning model further comprises an actual finding for the sample.
15. The non-transitory CRM of claim 10 , comprising computer readable program code,
which when executed by the computer processor, further enables the computer processor to:
prior to deploying the first learning model:
obtain a default model state;
initialize the first learning model with the default model state; and
adjust the first learning model using a first record set to obtain the first learning model configured with the first adjusted model state.
16. The non-transitory CRM of claim 15 , comprising computer readable program code to select the second learning model, which when executed by the computer processor, further enables the computer processor to:
initialize the second learning model with the first adjusted model state;
adjust the second learning model using a second record set to obtain the second learning model configured with the second adjusted model state;
initialize a third learning model with the default model state;
adjust the third learning model using a third record set to obtain the third learning model configured with a third adjusted model state;
make a second determination that the second learning model outperforms the third learning model; and
select the second learning model based on the second determination.
17. The non-transitory CRM of claim 16 , wherein the second record set excludes the first record set, wherein the third record set comprises the first and second record sets.
18. The non-transitory CRM of claim 15 , comprising computer readable program code to select the second learning model, which when executed by the computer processor, further enables the computer processor to:
initialize a third learning model with the first adjusted model state;
adjust the third learning model using a second record set to obtain the third learning model configured with a third adjusted model state;
initialize the second learning model with the default model state;
adjust the second learning model using a third record set to obtain the second learning model configured with the second adjusted model state;
make a second determination that the second learning model outperforms the third learning model; and
select the second learning model based on the second determination.