Systems and methods for risk factor predictive modeling with dynamic training
View Patent ↗A system and method for dynamic model training of a predictive machine learning model accesses data points of a training dataset including a plurality of model covariates. The predictive machine learning model is configured to generate an output including a risk rank representative of a mortality risk. The method selects one of the covariates and generates a historical data distribution for the selected covariate by applying the model to the training dataset including a plurality of historical application records. The method determines a current data distribution for the selected covariate. When comparison of the current data distribution with the historical data distribution indicates a data distribution shift exceeding a predetermined threshold, the method automatically updates parameters of the predictive machine learning model and retrains the predictive machine learning model using the updated parameters. Comparison of the current data distribution with the historical data distribution may employ covariate shift adaptation.
1. A computer-implemented method for training predictive machine learning models for mitigating drift by dynamically updating quantitative contributions of covariate data comprising:
accessing, by a processor, a training dataset comprising data points including a plurality of covariates of a predictive machine learning model, wherein the predictive machine learning model is configured to generate predictions for the plurality of covariates and a net classification based on quantitative contributions of the plurality of covariates, wherein the training dataset comprises a plurality of historical application records associated with time metrics failing within a time period;
selecting, by the processor, one of the plurality of covariates representing a health characteristic of the predictive machine learning model, wherein each covariate is associated with one or more indicators;
selecting, by the processor, an indicator associated with the selected covariate;
generating, by the processor, a historical data distribution for selected covariate over a range of the selected indicator by applying the predictive machine learning model to the training dataset;
determining, by the processor, a current data distribution for the selected covariate over the range of the selected indicator by applying the predictive machine learning model to a test dataset comprising data points associated with time metrics falling within a recent time period;
when a comparison of the current data distribution with the historical data distribution indicates a change in data patterns exceeding a predetermined threshold,
automatically updating, by the processor, one or more parameters of the predictive machine learning model by reassigning a quantitative contribution of the selected covariate; and
training, by the processor, the predictive machine learning model to generate the net classification with the reassigned quantitative contribution of the selected covariate using the training dataset of historical application records.
2. The method of claim 1 , wherein the comparing step calculates a covariate shift of the current data distribution relative to the historical data distribution.
3. The method of claim 2 , wherein the updating one or more parameters of the predictive machine learning model comprises eliminating the selected covariate from the predictive machine learning model.
4. The method of claim 2 , wherein the updating one or more parameters of the predictive machine learning model comprises eliminating the selected covariate from the predictive machine learning model if the selected covariate is not a high importance feature of the predictive machine learning model.
5. The method of claim 1 , wherein the updating one or more parameters of the predictive machine learning model comprises applying a temporal adjustment to the selected covariate in each of the plurality of historical application records, wherein the temporal adjustment is based on the time metric of each of the historical application records.
6. The method of claim 1 , wherein the predetermined threshold comprises a value of temporal drift of the current data distribution relative to the historical data distribution.
7. The method of claim 1 , wherein the plurality of historical application records represent policies issued from the historical application records, and the time metric comprises a policy date.
8. The method of claim 1 , wherein the plurality of historical application records represent policies declined or not taken from the historical application records, and the time metric comprises an application date.
9. The method of claim 1 , wherein the classification comprises an assignment to one of several classification groups based upon an algorithmic rule for assigning to one of several classification groups based on a model prediction for the selected covariate representing the health characteristic.
10. The method of claim 9 , wherein the updating one or more parameters of the predictive machine learning model comprises updating the algorithmic rule to reassign to one of the several classification groups based on the model prediction for the selected covariate representing the health characteristic.
11. The method of claim 1 , wherein the predictive machine learning model is a probabilistic model, and wherein the historical data distribution and the current data distribution are probability distributions.
12. The method of claim 1 , wherein the method further comprises:
generating, by the processor, at least one covariate contribution corresponding to the plurality of covariates based on the predictions for the plurality of covariates and the quantitative contributions of the plurality of covariates; and
presenting, by the processor to a user device, a graphical user interface with graphical indicators displaying an impact of each covariate contribution on the net classification, wherein the net classification is determined as a combination of the covariate contributions.
13. A system for training predictive machine learning models for mitigating drift by dynamically updating quantitative contributions of covariate data comprising:
an analytical engine server containing a processor configured to execute a plurality of non-transitory computer-readable instructions configured to:
access a training dataset comprising data points including a plurality of covariates of a predictive machine learning model, wherein the predictive machine learning model is configured to generate predictions for the plurality of covariates and a net classification representative of a mortality risk based on quantitative contributions of the plurality of covariates, wherein the training dataset comprises a plurality of historical application records associated with time metrics failing within a time period;
select one of the plurality of covariates representing a health characteristic of the predictive machine learning model, wherein each covariate is associated with one or more indicators;
select an indicator associated with the selected covariate;
generate a historical data distribution for selected covariate over a range of the selected indicator by applying the predictive machine learning model to the training dataset;
determine a current data distribution for the selected covariate over the range of the selected indicator by applying the predictive machine learning model to a test dataset comprising data points associated with time metrics falling within a recent time period;
when a comparison of the current data distribution with the historical data distribution indicates a change in data patterns exceeding a predetermined threshold,
automatically update one or more parameters of the predictive machine learning model by reassigning a quantitative contribution of the selected covariate; and
train the predictive machine learning model to generate the net classification with the reassigned quantitative contribution of the selected covariate using the training dataset of historical application records.
14. The system of claim 13 , wherein the comparison calculates a covariate shift of the current data distribution relative to the historical data distribution.
15. The system of claim 13 , wherein the automatic update applies a temporal adjustment to the selected covariate in each of the plurality of historical application records, wherein the temporal adjustment is based on the time metric of each of the historical application records.
16. The system of claim 13 , wherein the predetermined threshold comprises a value of temporal drift of the current data distribution relative to the historical data distribution.
17. The system of claim 13 , wherein the analytical engine server is further configured to:
generate at least one covariate contribution corresponding to the plurality of covariates based on the predictions for the plurality of covariates and the quantitative contributions of the plurality of covariates; and
present a graphical user interface to a user device with graphical indicators displaying an impact of each covariate contribution on the net classification, wherein the net classification is determined as a combination of the covariate contributions.