SYSTEMS AND METHODS TO SCREEN A PREDICTIVE MODEL FOR RISKS OF THE PREDICTIVE MODEL
Systems and methods to screen a predictive model for risks of the predictive model are provided. The method includes obtaining a predictive model and metadata of the predictive model. The method also includes determining, based on a set of criteria for screening the predictive model, a risk of one or more negative consequences associated with the predictive model. The method further includes providing an analysis of the risk of one or more negative consequences.
1 . A computer-implemented method to screen a predictive model for risks of the predictive model, comprising:
obtaining a predictive model and metadata of the predictive model;
determining, based on a set of criteria for screening the predictive model, a risk of one or more negative consequences associated with the predictive model; and
generating an analysis of the risk of one or more negative consequences.
2 . The computer-implemented method of claim 1 , wherein a criterion of the set of criteria is a model complexity of the predictive model, and wherein determining, based on the set of criteria for screening the metadata comprises determining, based on the model complexity of the predictive model, the risk of the one or more negative consequences.
3 . The computer-implemented method of claim 1 , wherein a criterion of the set of criteria is a variability of the predictive model across a plurality of stratifications, and wherein determining, based on the set of criteria for screening the metadata comprises determining, based on the variability of the predictive model across the plurality of stratifications, the risk of the one or more negative consequences.
4 . The computer-implemented method of claim 1 , wherein a criterion of the set of criteria is a presence of a predetermined flag associated with the predictive model, and wherein determining, based on the set of criteria for screening the metadata comprises determining, based on the predetermined flag associated with the predictive model, the risk of the one or more negative consequences.
5 . The computer-implemented method of claim 1 , wherein a criterion of the set of criteria is a user-defined rule associated with the predictive model, and wherein determining, based on the set of criteria for screening the metadata comprises determining, based on the user-defined rule associated with the predictive model, the risk of the one or more negative consequences.
6 . The computer-implemented method of claim 1 , wherein a criterion of the set of criteria is a similarity in a characteristic of interest, and wherein determining, based on the set of criteria for screening the metadata comprises determining, based on the similarity in the characteristic of interest, the risk of the one or more negative consequences.
7 . The computer-implemented method of claim 1 , further comprising updating the metadata of the predictive model to account for the risk of the one or more negative consequences.
8 . The computer-implemented method of claim 1 , further comprising generating one or more empirical indicators of the risk of the one or more negative consequences, wherein the analysis comprises the one or more empirical indicators.
9 . The computer-implemented method of claim 8 , further comprising determining a number of negative feedbacks of the predictive model, wherein the number of negative feedbacks of the predictive model is an empirical indicator of the one or more empirical indicators.
10 . The computer-implemented method of claim 1 , further comprising providing the analysis of the risk of the one or more negative consequences for display on a display screen of an electronic device.
11 . The computer-implemented method of claim 1 , further comprising:
determining whether the risk of the one or more negative consequences is greater than a threshold to suggest a modification to the predictive model; and
in response to a determination that the risk of the one or more negative consequences is greater than the threshold, including an indication of whether the risk of the one or more negative consequences is greater than the threshold and a suggestion to modify the predictive model to reduce the risk of the one or more negative consequences in the analysis.
12 . The computer-implemented method of claim 11 , further comprising:
determining, based on the risk of the one or more negative consequences, an update to the metadata of the predictive model; and
updating the metadata of the predictive model to account for the risk of the one or more negative consequences.
13 . A predictive model screening system, comprising:
a storage medium; and
one or more processors configured to:
obtain a predictive model and metadata of the predictive model;
determine, based on a set of criteria for screening the predictive model, a risk of one or more negative consequences associated with the predictive model;
generate an analysis of the risk of the one or more negative consequences; and
provide the analysis of the risk of the one or more negative consequences for display on a display screen of an electronic device.
14 . The predictive model screening system of claim 13 , wherein the one or more processors are further configured to periodically update the metadata of the predictive model to account for the risk of the one or more negative consequences associated with the predictive model.
15 . The predictive model screening system of claim 13 , wherein the one or more processors are further configured to generate one or more empirical indicators of the risk of the one or more negative consequences, wherein the analysis comprises the one or more empirical indicators.
16 . The predictive model screening system of claim 15 , wherein the one or more processors are further configured to determine a number of negative feedbacks of the predictive model, wherein the number of negative feedbacks of the predictive model is an empirical indicator of the one or more empirical indicators.
17 . The predictive model screening system of claim 13 , wherein the one or more processors are further configured to:
determine whether the risk of the one or more negative consequences is greater than a threshold to suggest a modification to the predictive model;
in response to a determination that the risk of the one or more negative consequences is greater than the threshold, include an indication of whether the risk of the one or more negative consequences is greater than the threshold and a suggestion to modify the predictive model to reduce the risk of the one or more negative consequences in the analysis;
determine, based on the risk of the one or more negative consequences, an update to the metadata of the predictive model; and
update the metadata of the predictive model to account for the risk of the one or more negative consequences.
18 . A non-transitory computer-readable medium comprising instructions, which when executed by a processor, cause the processor to perform operations comprising:
obtaining a predictive model and metadata of the predictive model;
determining, based on a set of criteria for screening the predictive model, a risk of one or more negative consequences associated with the predictive model;
generating one or more empirical indicators of the risk of the one or more negative consequences;
generating an analysis of the risk of the one or more negative consequences, wherein the analysis comprises the one or more empirical indicators; and
providing the analysis of the risk of the one or more negative consequences for display on a display screen of an electronic device.
19 . The non-transitory computer-readable medium of claim 18 , wherein the instruction, when executed by the processor, cause the processor to perform operations comprising:
determining, based on the risk of the one or more negative consequences, an update to the metadata of the predictive model; and
updating the metadata of the predictive model to account for the risk of the one or more negative consequences associated with the predictive model.
20 . The non-transitory computer-readable medium of claim 18 , wherein the instruction, when executed by the processor, cause the processor to perform operations comprising:
determining whether the risk of the one or more negative consequences is greater than a threshold to suggest a modification to the predictive model; and
in response to a determination that the risk of the one or more negative consequences is greater than the threshold, including an indication of whether the risk of the one or more negative consequences is greater than the threshold and a suggestion to modify the predictive model to reduce the risk of the one or more negative consequences in the analysis;
determining, based on the risk of the one or more negative consequences, an update to the metadata of the predictive model; and
updating the metadata of the predictive model to account for the risk of the one or more negative consequences.