Systems and methods of implementing scorecards and boosted decision trees
Systems and methods are described for machine learning-based generation of scorecards and boosted decision trees that facilitate explainable predictions. A scorecard machine learning model may be applied to historical records such that the model, for each of a number of variables, automatically generates (a) normal bins for normal values of the variable that fall within a valid range of values and (b) at least one special bin for special values of the variable that fall outside the valid range of values. Adjacent bins of the normal bins may be separated by a threshold value and each normal bin and each special bin may have an assigned score value. A risk assessment score may be generated based at least in part on the model identifying the assigned value score for each of the variables based on the normal or special bin to which each variable is assigned.
1 . A system for generating risk assessment scores, the system comprising:
a data store including historical records regarding each of a number of individuals; and
at least one physical processor programmed by instructions in a memory to:
receive, from a user system, a request for a risk assessment score for an individual;
based on parsing the received request, identify an identifier for the individual;
based on the identifier, obtain, from the data store, one or more historical records for the individual;
based on the received request, select a corresponding machine learning (ML) model, the selected ML model configured to manage a plurality of variables for use in generating the risk assessment score, wherein at least a subset of the plurality of variables are associated with information in the one or more historical records for the individual;
apply the selected ML model to the one or more historical records, wherein the selected ML model is trained using a gradient boosted decision tree comprising one-split stumps, one or more of the one-split stumps being constrained by monotonicity rules so that the selected ML model produces a transparent and interpretable scorecard representation, wherein the selected ML model is a scorecard ML model that, for each of the variables of the plurality of variables, automatically generates:
(a) normal bins for normal values of a variable of the variables that fall within a valid range of values for the variable; and
(b) at least one special bin for special values of the variable that fall outside the valid range of values for the variable, wherein adjacent bins of the normal bins are separated by a threshold value and each normal bin and each special bin have an assigned score value;
based on the selected ML model, generate the risk assessment score for the individual, wherein the selected ML model generates the risk assessment score based at least in part on identifying the assigned value score for each of the plurality of variables based on the normal or special bin to which each individual variable of the plurality of variables is assigned;
generate an output comprising the risk assessment score; and
cause transmission, to the user system, of the output.
2 . The system of claim 1 , wherein the risk assessment score represents a quantitative estimate of a probability that the individual will display a defined behavior as determined by the scorecard ML model based on identification of the normal or special bin to which each individual variable of the plurality of variables is assigned.
3 . The system of claim 1 , wherein the monotonicity rules are applied on the normal bins but not on the at least one special bin, where the monotonicity rules limit to one of non-increasing or non-decreasing changes in weights for each adjacent subsequent bin of the normal bins.
4 . The system of claim 3 , wherein the instructions further cause the physical processor to:
determine an explanation for the generated risk assessment score, wherein the explanation is determined based at least in part on identification of at least one of the normal bins to which an individual variable is assigned in view of the monotonicity rules, wherein the output further comprises the explanation for the generated risk assessment score.
5 . The system of claim 1 , wherein the instructions further cause the physical processor to implement binarsity regularization that causes the scorecard ML model to penalize a difference in weights associated with at least one pair of adjacent normal bins.
6 . The system of claim 5 , wherein the scorecard ML model combines two or more neighboring bins based at least in part on the binarsity regularization.
7 . The system of claim 1 , wherein automatically generating the normal bins comprises applying one of equal-sized binning or quantile-based binning within the valid range of values for the variable.
8 . The system of claim 1 , wherein the instructions further cause the physical processor to implement a U-shaped constraint for at least one variable, wherein implementing the U-shaped constraint comprises the scorecard ML model splitting the at least one variable into two virtual variables or sub-variables that are separately analyzed by the scorecard ML model.
9 . The system of claim 1 , wherein the processor is further configured to, prior to causing transmission:
generate, together with the risk assessment score, an explanation identifying the variables and bins that contributed to the score; and
update the output to include the explanation.
10 . A computer-implemented method for optimizing binning in a scorecard machine learning (ML) model, the computer-implemented method comprising:
receiving, from a user system, a request for a risk assessment score for an individual;
based on parsing the received request, identifying an identifier for the individual;
based on the identifier, obtaining, from a networked data store, one or more historical records for the individual;
identifying, for a first ML model to be applied to generate the risk assessment score, a variable for which a plurality of bins are to be generated, the variable having a valid range within which normal bins of the plurality of bins are defined and outside of which at least one special bin of the plurality of bins is defined;
applying a second ML model to the variable based on generating at least one split value for the valid range, wherein generating each of the at least one split value comprises generating a single level decision tree for the split value and enforcing monotonicity with respect to the generated single level decision for each of the at least one split value in the valid range;
generating the normal bins for the valid range based at least in part on the generated at least one split value;
applying a binarsity regularization term that penalizes large differences in weights between adjacent bins, so that bins with substantially similar weights are merged together to improve stability of the scorecard ML model;
generating the at least one special bin to include one or more special values of the variable;
generating the risk assessment score based on applying the first ML model to the one or more historical records using the generated normal bins and the generated at least one special bin;
generating an output comprising the generated risk assessment score; and
cause transmission, to the user system, of the output.
11 . The computer-implemented method of claim 10 , wherein the single level decision tree comprises a one-split tree with only two leaf nodes.
12 . The computer-implemented method of claim 10 , wherein the second ML model is trained at least in part by utilizing a gradient boosting decision tree algorithm employing stumps.
13 . The computer-implemented method of claim 12 , wherein the second ML model enforces monotonicity constraints on the stumps, wherein enforcing the monotonicity constraints on the stumps comprises ignoring candidate splits on which a monotonicity constraint is violated.
14 . The computer-implemented method of claim 10 further comprising treating multiple special values as a categorical variable.
15 . The computer-implemented method of claim 14 further comprising enabling a plurality of special values falling within the categorical variable to be placed on either a left side or right side of a split point in a tree.
16 . A computer-implemented method for generating risk assessment scores, the computer-implemented method comprising:
receiving, from a user system, a request for a risk assessment score for an individual;
based on parsing the received request, identifying an identifier for the individual;
based on the identifier, obtaining one or more historical records for the individual;
selecting a corresponding machine learning (ML) model, the selected ML model configured to manage a plurality of variables for use in generating the risk assessment score, wherein at least a subset of the plurality of variables are associated with information in the one or more historical records for the individual;
applying the selected ML model to the one or more historical records, wherein the selected ML model is a scorecard ML model trained using a gradient boosted decision tree built from one-split stumps, one or more of the one-split stumps being constrained by monotonicity rules so that the selected ML model produces a transparent and interpretable scorecard representation, wherein the scorecard ML model that, for each of the variables of the plurality of variables, automatically generates:
(a) normal bins for normal values of a variable of the variables that fall within a valid range of values for the variable; and
(b) at least one special bin for special values of the variable that fall outside the valid range of values for the variable, wherein adjacent bins of the normal bins are separated by a threshold value and each normal bin and each special bin have an assigned score value;
generating the risk assessment score for the individual based on the selected ML model, wherein the selected ML model generates the risk assessment score based at least in part on identifying the assigned value score for each of the plurality of variables based on the normal or special bin to which each individual variable of the plurality of variables is assigned;
generating an output comprising the risk assessment score; and
causing transmission, to the user system, of the output.
17 . The computer-implemented method of claim 16 , wherein the monotonicity rules are applied on the normal bins but not on the at least one special bin, where the monotonicity rules limit to one of non-increasing or non-decreasing changes in weights for each adjacent subsequent bin.
18 . The computer-implemented method of claim 16 further comprising:
determining an explanation for the generated risk assessment score, wherein the explanation is determined based at least in part on identification of at least one of the normal bins to which an individual variable is assigned in view of the monotonicity rules, wherein the output further comprises the explanation for the generated risk assessment score.
19 . The computer-implemented method of claim 16 , wherein generating the normal bins comprises applying one of equal-sized binning or quantile-based binning within the valid range of values for the variable.
20 . The computer-implemented method of claim 16 further comprising implementing a U-shaped constraint for at least one variable, wherein implementing the U-shaped constraint comprises the scorecard ML model splitting the at least one variable into two virtual variables or sub-variables that are separately analyzed by the scorecard ML model.