Explainable complex model
Certain aspects of the present disclosure provide techniques for generating a human readable summary explanation to a user for an outcome generated by a complex machine learning model. In one embodiment, a risk assessment service can receive a request from a user in which a risk model of the risk assessment service performs a specific task (e.g., determining the level of risk associated with the user). Once the risk model determines the risk associated with the user, in order to comply with regulations from a compliance system, the risk model can provide a user with an explanation as to the outcome for transparency purposes.
1. A computer-implemented method, comprising:
receiving a compliance regulation from a compliance system, wherein the compliance regulation includes a mapping of at least one feature from a set of features to at least one human-readable explanation;
accessing a set of user data from one or more user accounts;
extracting the set of features from the set of user data corresponding to user risk activity;
generating, via a risk model, an attribution value for each feature of the set of features, wherein:
the risk model comprises a probabilistic model trained on training data including a Weight of Evidence value calculated for each feature of the set of features,
the Weight of Evidence value calculated for each feature corresponds to an indication of whether each feature of the set of features causes an increase or a decrease to risk scores generated by the risk model, and
a directionality of changes to risk scores generated by the risk model is constrained by the Weight of Evidence value calculated for each feature of the set of features;
generating, based on the set of features, a risk score corresponding to the user activity via the risk model;
determining the risk score does not meet a pre-determined threshold; and
generating a human-readable explanation indicating a reason that the risk score does not meet the pre-determined threshold, the generating of the human-readable explanation comprising:
determining, from the set of features, a feature with a highest attribution value; and
selecting the human-readable explanation based on a mapping of the human-readable explanation to the feature with the highest attribution value.
2. The computer-implemented method of claim 1 , wherein the risk score corresponds to a predicted outcome associated with the user.
3. The computer-implemented method of claim 2 , wherein the predicted outcome is one of:
a negative predicted outcome if the risk score does not meet the pre-determined threshold; or
a positive predicted outcome if the risk score meets the pre-determined threshold.
4. The computer-implemented method of claim 1 , wherein the risk model is trained with:
historical user data from a set of users;
historical risk scores for the set of users; and
historical actual outcomes associated with the set of users.
5. The computer-implemented method of claim 1 , wherein the risk model is a XGBoost non-linear risk model.
6. The computer-implemented method of claim 5 , wherein the XGBoost non-linear risk model includes monotonic constraints.
7. The computer-implemented method of claim 1 , further comprising:
receiving feedback from the user based on the human-readable explanation; and
including the feedback in training the risk model.
8. The computer-implemented method of claim 1 , wherein the extraction of the set of features further comprises transforming the set of user data into a set of categories is based on associating each user data in the set of user data with a respective category.
9. A system, comprising:
a memory having executable instructions stored thereon; and
a processor configured to execute the executable instructions in order to cause the system to:
receive a compliance regulation from a compliance system, wherein the compliance regulation includes a mapping of at least one feature from a set of features to at least one human-readable explanation;
access a set of user data from one or more user accounts;
extract the set of features from the set of user data corresponding to user risk activity;
generate, via a risk model, an attribution value for each feature of the set of features, wherein:
the risk model comprises a probabilistic model trained on training data including a Weight of Evidence value calculated for each feature of the set of features,
the Weight of Evidence value calculated for each feature corresponds to an indication of whether each feature of the set of features causes an increase or a decrease to risk scores generated by the risk model, and
a directionality of changes to risk scores generated by the risk model is constrained by the Weight of Evidence value calculated for each feature of the set of features;
generate, based on the set of features, a risk score corresponding to the user activity via the risk model;
determine the risk score does not meet a pre-determined threshold; and
generate a human-readable explanation indicating a reason that the risk score does not meet the pre-determined threshold, wherein in order to generate the human-readable explanation, the processor is configured to cause the system to:
determine, from the set of features, a feature with a highest attribution value; and
select the human-readable explanation based on a mapping of the human-readable explanation to the feature with the highest attribution value.
10. The system of claim 9 , wherein the risk score corresponds to a predicted outcome associated with the user.
11. The system of claim 10 , wherein the predicted outcome is one of:
a negative predicted outcome if the risk score does not meet the pre-determined threshold; or
a positive predicted outcome if the risk score meets the pre-determined threshold.
12. The system of claim 9 , wherein the risk model is trained with:
historical user data from a set of users;
historical risk scores for the set of users; and
historical actual outcomes associated with the set of users.
13. The system of claim 9 , wherein the risk model is a XGBoost non-linear risk model.
14. The system of claim 13 , wherein the XGBoost non-linear risk model includes monotonic constraints.
15. The system of claim 9 , wherein the processor is further configured to cause the system to:
receive feedback from the user based on the human-readable explanation; and
include the feedback in training the risk model.
16. The system of claim 9 , wherein the transformation of the set of user data into the set of categories is based on associating each user data in the set of user data with a respective category.