Generating gradient boosting machine model reason codes
A method for constructing a lookup table for determining outcomes of a prediction model. The method includes: (a) determining all possible outcomes for each hierarchy of the prediction model; (b) determining contributions of a set of features for each hierarchy of the prediction model; and (c) constructing the lookup table based on all possible outcomes and contributions of each of the set of features.
1 . A method carried out by one or more computing servers associated with a financial services company, the method comprising:
applying, by the one or more computing servers, a machine learning process to training data and thereby training a gradient boosting machine (GBM) model that is configured to (i) evaluate data for a set of features that is related to extending a financial service to an individual and (ii) based on the evaluation, output a prediction related to extending the financial service to the individual, wherein the trained GBM model comprises a combination of decision trees that each have a respective set of leaf nodes representing a respective set of available tree-specific outcomes for the decision tree;
evaluating, by the one or more computing servers, each respective decision tree in the GBM model's combination of decision trees by:
traversing the respective decision tree from its parent node to each leaf node in its respective set of leaf nodes and thereby identifying the respective set of available tree- specific outcomes for the respective decision tree; and
for each available tree-specific outcome in the respective decision tree's set of available tree-specific outcomes, evaluating how the GBM model's set of features contributes to the outcome and thereby determining an outcome-specific subset of feature contribution data corresponding to each available tree-specific outcome in the respective set of available tree-specific outcomes for the respective decision tree;
constructing, by the one or more computing servers, a lookup table based on the respective set of available tree-specific outcomes and the corresponding outcome-specific subsets of feature contribution data that are determined for each respective decision tree in the GBM model's combination of decision trees, wherein the lookup table comprises (i) a first dimension corresponding to the available tree-specific outcomes of the GBM model and (ii) a second dimension corresponding to the GBM model's set of features;
after training the GBM model and constructing the lookup table:
executing, by the one or more computing servers, the GBM model by inputting, into the GBM model, a given feature dataset for the set of features that is related to extending the financial service to a given individual, wherein the executed GBM model functions to (i) receive the given feature dataset as input, (ii) evaluate the given feature dataset using the GBM model's combination of decision trees and thereby determine a given set of tree-specific outcomes, and (iii) based on the given set of tree-specific outcomes determined by the GBM model's combination of decision trees, output a given prediction related to extending the financial service to the given individual;
based on the given prediction related to extending the financial service to the given individual that is output by the executed GBM model, making a decision, by the one or more computing servers, of whether to extend the financial service to the given individual; and
after making the decision of whether to extend the financial service to the given individual:
determining, by the one or more computing servers, one or more reasons for the decision without re-executing the GBM model by: (i) looking up, in the lookup table, the given set of tree-specific outcomes that were determined by the GBM model's combination of decision trees during execution, (ii) retrieving, from the lookup table, the outcome-specific subsets of feature contribution data corresponding to the looked-up tree-specific outcomes in the given set, (iii) aggregating the retrieved, outcome-specific subsets of feature contribution data corresponding to the looked-up tree-specific outcomes into an aggregated set of feature contribution data that indicates respective contributions of the GBM model's set of features to the given prediction output by the GBM model during execution, (iv) based on the aggregated set of feature contribution data, identifying one or more features that influenced the given prediction output by the executed GBM model, and (v) identifying one or more reason codes that correspond to the identified one or more features, wherein the identified one or more reason codes indicate the one or more reasons for the decision; and
outputting, by the one or more computing servers, an indication of (i) the decision of whether to extend the financial service to the given individual and (ii) the one or more reasons for the decision.
2 . The method of claim 1 , wherein the first dimension of the lookup table comprises the rows of the lookup table and the second dimension of the lookup table comprises the columns of the lookup table.
3 . The method of claim 1 , wherein, for each available tree- specific outcome in the respective set of available tree-specific outcomes for each respective decision tree, determining the outcome-specific subset of feature contribution data comprises:
determining node values of the respective decision tree corresponding to the available tree-specific outcome; and
based on the node values, determining at least one respective contribution value for each of at least a subset of the set of features.
4 . The method of claim 1 , wherein the lookup table is static.
5 . The method of claim 1 , wherein the aggregated set of feature contribution data comprises, for each respective feature in the set of features, one or both of (i) a respective total contribution value for the respective feature in the set of features or (ii) a respective net contribution value for the respective feature in the set of features.
6 . A computing system associated with a financial services company, the computing system comprising:
a processor; and
a non-transitory computer readable medium storing processor-executable instructions thereon that, when executed by the processor, cause the computing system to perform functions comprising:
applying a machine learning process to training data and thereby training a gradient boosting machine (GBM) model that is configured to (i) evaluate data for a set of features that is related to extending a financial service to an individual and (ii) based on the evaluation, output a prediction related to extending the financial service to the individual, wherein the trained GBM model comprises a combination of decision trees that each have a respective set of leaf nodes representing a respective set of available tree-specific outcomes for the decision tree;
evaluating each respective decision tree in the GBM model's combination of decision trees by:
traversing the respective decision tree from its parent node to each leaf node in its respective set of leaf nodes and thereby identifying the respective set of available tree-specific outcomes for the respective decision tree; and
for each available tree-specific outcome in the respective decision tree's set of available tree-specific outcomes, evaluating how the GBM model's set of features contributes to the outcome and thereby determining an outcome-specific subset of feature contribution data corresponding to each available tree-specific outcome in the respective set of available outcomes for the respective decision tree;
constructing a lookup table based on the respective set of available tree-specific outcomes and the corresponding outcome-specific subsets of feature contribution data that are determined for each respective decision tree in the GBM model's combination of decision trees, wherein the lookup table comprises (i) a first dimension corresponding to the available tree-specific outcomes of the GBM model and (ii) a second dimension corresponding to the GBM model's set of features; and
after training the GBM model and constructing the lookup table:
executing the GBM model by inputting, into the GBM model, a given feature dataset for the set of features that is related to extending the financial service to a given individual, wherein the executed GBM model functions to (i) receive the given feature dataset as input, (ii) evaluate the given feature dataset using the GBM model's combination of decision trees and thereby determine a given set of tree-specific outcomes, and (iii) based on the given set of tree-specific outcomes determined by the GBM model's combination of decision trees, output a given prediction related to extending the financial service to the given individual;
based on the given prediction related to extending the financial service to the given individual that is output by the executed GBM model, making a decision of whether to extend the financial service to the given individual; and
after making the decision of whether to extend the financial service to the given individual:
determining one or more reasons for the decision without re-executing the GBM model by: (i) looking up, in the lookup table, the given set of tree-specific outcomes that were determined by the GBM model's combination of decision trees during execution, (ii) retrieving, from the lookup table, the outcome-specific subsets of feature contribution data corresponding to the looked-up tree-specific outcomes in the given set, (iii) aggregating the retrieved, outcome-specific subsets of feature contribution data corresponding to the looked-up tree-specific outcomes into an aggregated set of feature contribution data that indicates respective contributions of the GBM model's set of features to the given prediction output by the GBM model during execution, (iv) based on the aggregated set of feature contribution data, identifying one or more features that influenced the given prediction output by the executed GBM model, and (v) identifying one or more reason codes that correspond to the identified one or more features, wherein the identified one or more reason codes indicate the one or more reasons for the decision; and
outputting an indication of (i) the decision of whether to extend the financial service to the given individual and (ii) the one or more reasons for the decision.
7 . The computing system of claim 6 , wherein the first dimension of the lookup table comprises the rows of the lookup table and the second dimension of the lookup table comprises the columns of the lookup table.
8 . The computing system of claim 6 , wherein the lookup table is static.
9 . The computing system of claim 6 , wherein the aggregated set of feature contribution data comprises, for each respective feature in the set of features, one or both of (i) a respective total contribution value for the respective feature in the set of features or (ii) a respective net contribution value for the respective feature in the set of features.
10 . The method of claim 1 , wherein, for each available tree-specific outcome in the respective set of available tree-specific outcomes for each respective decision tree, the corresponding outcome-specific subset of feature contribution data comprises at least one respective contribution value for each of at least a subset of the set of features.
11 . The method of claim 10 , wherein, for each available tree-specific outcome in the respective set of available tree-specific outcomes for each respective decision tree, the at least one respective contribution value for each of at least the subset of the set of features comprises a respective total contribution value for each of at least the subset of the set of features.
12 . The method of claim 10 , wherein, for each available tree- specific outcome in the respective set of available tree-specific outcomes for each respective decision tree, the at least one respective contribution value for each of at least the subset of the set of features comprises a first contribution value associated with a first directionality and a second contribution value associated with a second directionality for each of at least the subset of the set of features.
13 . The method of claim 1 , wherein the aggregate set of feature contribution data comprises, for each respective feature in the set of features, a respective contribution value of a first type for the respective feature in the set of features, and wherein identifying the one or more features that influenced the given prediction output by the executed GBM model comprises:
identifying any feature in the set of features having a respective contribution value of the first type that is above a threshold value.
14 . The method of claim 1 , wherein the aggregate set of feature contribution data comprises, for each respective feature in the set of features, a respective contribution value of a first type for each feature in the set of features, and wherein identifying the one or more features that influenced the given prediction output by the executed GBM model comprises:
based on the respective contribution values of the first type for the set of features, ranking the set of features; and
identifying a given number of features at a top of the ranked set of features.
15 . The computing system of claim 6 , wherein, for each available tree-specific outcome in the respective set of available tree-specific outcomes for each respective decision tree, the corresponding outcome-specific subset of feature contribution data comprises at least one respective contribution value for each of at least a subset of the set of features.
16 . The computing system of claim 15 , wherein, for each available tree-specific outcome in the respective set of available tree-specific outcomes for each respective decision tree, the at least one respective contribution value for each of at least the subset of the set of features comprises a respective total contribution value for each of at least the subset of the set of features.
17 . The computing system of claim 15 , wherein, for each available tree-specific outcome in the respective set of available tree-specific outcomes for each respective decision tree, the at least one respective contribution value for each of at least the subset of the set of features comprises a first contribution value associated with a first directionality and a second contribution value associated with a second directionality for each of at least the subset of the set of features.
18 . A non-transitory computer-readable medium, wherein the non-transitory computer-readable medium is provisioned with program instructions that, when executed by at least one processor, cause a computing system associated with a financial services company to:
apply a machine learning process to training data and thereby train a gradient boosting machine (GBM) model that is configured to (i) evaluate data for a set of features that is related to extending a financial service to an individual and (ii) based on the evaluation, output a prediction related to extending the financial service to the individual, wherein the trained GBM model comprises a combination of decision trees that each have a respective set of leaf nodes representing a respective set of available tree-specific outcomes for the decision tree;
evaluate each respective decision tree in the GBM model's combination of decision trees by:
traversing the respective decision tree from its parent node to each leaf node in its respective set of leaf nodes and thereby identifying the respective set of available tree- specific outcomes for the respective decision tree; and
for each available tree-specific outcome in the respective decision tree's set of available tree-specific outcomes, evaluating how the GBM model's set of features contributes to the outcome and thereby determining an outcome-specific subset of feature contribution data corresponding to each available tree-specific outcome in the respective set of available tree-specific outcomes for the respective decision tree;
construct a lookup table based on the respective set of available tree-specific outcomes and the corresponding outcome-specific subsets of feature contribution data that are determined for each respective decision tree in the GBM model's combination of decision trees, wherein the lookup table comprises (i) a first dimension corresponding to the available tree-specific outcomes of the GBM model and (ii) a second dimension corresponding to the GBM model's set of features; and
after training the GBM model and constructing the lookup table:
execute the GBM model by inputting, into the GBM model, a given feature dataset for the set of features that is related to extending the financial service to a given individual, wherein the executed GBM model functions to (i) receive the given feature dataset as input, (ii) evaluate the given feature dataset using the GBM model's combination of decision trees and thereby determine a given set of tree-specific outcomes, and (iii) based on the given set of tree-specific outcomes determined by the GBM model's combination of decision trees, output a given prediction related to extending the financial service to the given individual;
based on the given prediction related to extending the financial service to the given individual that is output by the executed GBM model, making a decision of whether to extend the financial service to the given individual; and
after making the decision of whether to extend the financial service to the given individual:
determine one or more reasons for the decision without re-executing the GBM model by: (i) looking up, in the lookup table, the given set of tree-specific outcomes that were determined by the GBM model's combination of decision trees during execution, (ii) retrieving, from the lookup table, the outcome-specific subsets of feature contribution data corresponding to the looked-up tree-specific outcomes in the given set, (iii) aggregating the retrieved, outcome-specific subsets of feature contribution data corresponding to the looked-up tree-specific outcomes into an aggregated set of feature contribution data that indicates respective contributions of the GBM model's set of features to the given prediction output by the GBM model during execution, (iv) based on the aggregated set of feature contribution data, identifying one or more features that influenced the given prediction output by the executed GBM model, and (v) identifying one or more reason codes that correspond to the identified one or more features, wherein the identified one or more reason codes indicate the one or more reasons for the decision; and
output an indication of (i) the decision of whether to extend the financial service to the given individual and (ii) the one or more reasons for the decision.
19 . The method of claim 1 , wherein the decision of whether to extend the financial service to the given individual comprises one of (i) a decision of whether to approve or deny the given individual's request for a new line of credit, (ii) a decision of whether to approve or deny the given individual's request for an increased credit limit on an existing line of credit, or (iii) a decision of whether to approve or deny a financial transaction initiated by the given individual.
20 . The computing system of claim 6 , wherein the decision of whether to extend the financial service to the given individual comprises one of (i) a decision of whether to approve or deny the given individual's request for a new line of credit, (ii) a decision of whether to approve or deny the given individual's request for an increased credit limit on an existing line of credit, or (iii) a decision of whether to approve or deny a financial transaction initiated by the given individual.