System and Method for Developing Proxy Models
A system and method for developing proxy models is provided. The system for developing proxy models comprising a proxy model development computer system in electronic communication with a training database storing training data therein, and a plurality of computer models including a complex model and a proxy model that are trained by the computer system using the training data from the training database, wherein the computer system evaluates performance of each of the plurality of computer models, and if the computer system determines that the proxy model at least meets pre-defined performance criteria and approximates performance of the complex model, then the computer system communicates to a user that the proxy model can substitute the complex model.
1 . A system for developing proxy models comprising:
a proxy model development computer system in electronic communication with a training database storing training data therein; and
a plurality of computer models including a complex model and a proxy model, each of the plurality of computer models trained by the computer system using the training data from the training database,
wherein the computer system evaluates performance of each of the plurality of computer models and, if the computer system determines that the proxy model meets pre-defined performance criteria and approximates performance of the complex model, then the computer system communicates to a user that the proxy model can be substituted for the complex model.
2 . The system of claim 1 , wherein the computer system trains the complex model using the training data and a target numeric score representing a target performance level.
3 . The system of claim 2 , wherein the computer system executes the complex model to generate a complex model score.
4 . The system of claim 3 , wherein the computer system trains a simple model using the training data and the target numeric score.
5 . The system of claim 4 , wherein the computer system executes the simple model to generate a simple model score.
6 . The system of claim 5 , wherein the computer system trains the proxy model using the training data and the complex model score.
7 . The system of claim 6 , wherein the computer system executes the proxy model to generate a proxy model score.
8 . The system of claim 7 , wherein the computer system determines whether to substitute the complex model with the proxy model by determining whether the proxy model approximates the complex model using an approximation test algorithm.
9 . The system of claim 8 , wherein the approximation test algorithm is the Kolmogorov-Smirnoff test.
10 . The system of claim 1 , wherein the training data used to train the complex model is a set of variables, and the training data used to train the proxy model is a subset of variables less than the set of variables.
11 . The system of claim 1 , wherein the proxy model is used to discern reason codes for model predictions.
12 . A method for developing proxy models, comprising the steps of:
electronically communicating by a proxy model development computer system with a training database storing training data therein;
training by the computer system a plurality of computer models including a complex model and a proxy model using the training data from the training database;
evaluating, by the computer system, performance of each of the plurality of computer models;
determining whether the proxy model at least meets pre-defined performance criteria and whether the proxy model approximates performance of the complex model; and
communicating to a user that the proxy model can be substituted for the complex model if the proxy model meets the pre-defined performance criteria and approximates performance of the complex model.
13 . The method of claim 12 , wherein the computer system trains the complex model using the training data and a target numeric score representing a target performance level.
14 . The method of claim 13 , further comprising executing the complex model to generate a complex model score.
15 . The method of claim 14 , wherein the computer system trains a simple model using the training data and the target numeric score.
16 . The method of claim 15 , further comprising executing the simple model to generate a simple model score.
17 . The method of claim 16 , wherein the computer system trains the proxy model using the training data and the complex model score.
18 . The method of claim 17 , further comprising executing the proxy model to generate a proxy model score.
19 . The method of claim 18 , wherein the computer system determines whether to substitute the complex model with the proxy model by determining whether the proxy model approximates the complex model using an approximation test algorithm.
20 . The method of claim 19 , wherein the approximation test algorithm is the Kolmogorov-Smirnoff test.
21 . The method of claim 12 , wherein the training data used to train the complex model is a set of variables, and the training data used to train the proxy model is a subset of variables less than the set of variables.
22 . The method of claim 12 , further comprising executing the proxy model to discern reason codes for model predictions.
23 . A computer-readable medium having computer-readable instructions stored thereon which, when executed by a computer system, cause the computer system to perform the steps of:
electronically communicating by a proxy model development computer system with a training database storing training data therein;
training by the computer system a plurality of computer models including a complex model and a proxy model using the training data from the training database;
evaluating, by the computer system, performance of each of the plurality of computer models;
determining whether the proxy model at least meets pre-defined performance criteria and whether the proxy model approximates performance of the complex model; and
communicating to a user that the proxy model can be substituted for the complex model if the proxy model meets the pre-defined performance criteria and approximates performance of the complex model.
24 . The computer-readable medium of claim 23 , wherein the computer system trains the complex model using the training data and a target numeric score representing a target performance level.
25 . The computer-readable medium of claim 24 , further comprising executing the complex model to generate a complex model score.
26 . The computer-readable medium of claim 25 , wherein the computer system trains a simple model using the training data and the target numeric score.
27 . The computer-readable medium of claim 26 , further comprising executing the simple model to generate a simple model score.
28 . The computer-readable medium of claim 27 , wherein the computer system trains the proxy model using the training data and the complex model score.
29 . The computer-readable medium of claim 28 , further comprising executing the proxy model to generate a proxy model score.
30 . The computer-readable medium of claim 29 , wherein the computer system determines whether to substitute the complex model with the proxy model by determining whether the proxy model approximates the complex model using an approximation test algorithm.
31 . The computer-readable medium of claim 30 , wherein the approximation test algorithm is the Kolmogorov-Smirnoff test.
32 . The computer-readable medium of claim 23 , wherein the training data used to train the complex model is a set of variables, and the training data used to train the proxy model is a subset of variables less than the set of variables.
33 . The computer-readable medium of claim 23 , further comprising executing the proxy model to discern reason codes for model predictions.