Information system providing explanation of models
A health care information system generates information that describes how different inputs to a model affect the output of the model, by creating a localized model for a given entity, and determining how the output of the localized model for the given entity changes in response to different inputs. The computer system builds the localized model for a given entity based on the model and on data values for the given entity. The computer system inputs one or more different data values for selected input features of the localized model, while data values for the remaining input features of the localized model are fixed to data values for the given entity, and obtains corresponding outputs from the localized model. The results from this localized model for the given entity indicate which of the selected input features have the most impact on the output of the model for that entity.
1 . A computer system, comprising:
a. a processing system comprising a processing device and computer storage, wherein the computer storage stores:
i. a first data structure including, for a plurality of entities, data values for input features for a trained computational model, wherein the data values for each entity are derived from a respective record for the entity in a data set, and
ii. a second data structure including data identifying a selected subset of the input features for the trained computational model, the selected subset having fewer features than the input features;
b. computer program code implementing the trained computational model which, when processed by the processing system, configures the processing system to apply input data values for the input features of a given entity to the trained computational model to output a predicted outcome for the given entity in response to the input data values for the input features of the entity;
c. computer program code that, when processed by the processing system, configures the processing system to generate a localized model for the given entity, wherein the localized model is distinct from and simplified with respect to the trained computational model and approximates the trained computational model, wherein the processing system generates the localized model based on a. the trained computational model and b. data values for the given entity from the first data structure for input features other than the selected subset of the input features identified by the second data structure, wherein the localized model has inputs and an output, wherein the inputs of the localized model correspond to only the selected subset of input features identified by the second data structure and the output of the localized model provides a predicted outcome for the given entity;
d. a sensitivity analysis module comprising computer program code that, when processed by the processing system, configures the processing system to analyze the localized model for the given entity by:
applying a plurality of different data values for the input features identified in the selected subset of the input features to the inputs of the localized model for the given entity, wherein the plurality of different data values are different from the data values for the selected subset of input features for the given entity as stored in the first data structure, whereby the localized model outputs respective predicted outcomes for the given entity for the different data values, and
storing, in the computer storage, the respective predicted outcomes output from the localized model for the given entity; and
e. a graphical user interface comprising computer program code that, when processed by the processing system, is responsive to the sensitivity analysis module to provide an output including human-understandable content describing how data values for the selected subset of input features likely would affect the predicted outcome of the trained computational model as applied to the given entity based on the stored respective predicted outcomes output from the localized model.
2 . The computer system of claim 1 , wherein the data set comprises health care information for a plurality of patients, wherein each patient in the plurality of patients has a respective record in the data set, and wherein the given entity is a given patient in the plurality of patients.
3 . The computer system of claim 1 , wherein the data set comprises health care information for a plurality of health care providers, wherein each health care provider in the plurality of health care providers has a respective record in the data set, and wherein the given entity is a given health care provider in the plurality of health care providers.
4 . The computer system of claim 1 , wherein the trained computational model is trained using a training set derived from the data set.
5 . The computer system of claim 1 , wherein the trained computational model performs classification of the entities into categories.
6 . The computer system of claim 1 , wherein the trained computational model computes risk factors associated with the entities.
7 . The computer system of claim 1 , wherein the entities are patients and the trained computational model computes predictions of outcomes for patients based on the records for the patients in the data set.
8 . The computer system of claim 1 , wherein the entities are patients and the trained computational model computes outcome scores for the patients based on the records for the patients in the data set.
9 . The computer system of claim 8 , wherein outcome scores are represented using an integer in a range of integer values.
10 . The computer system of claim 1 , wherein the entities are patients and the trained computational model computes factor scores for the patients based on the records for the patients in the data set.
11 . The computer system of claim 1 , wherein the second data structure comprises a library stored in the computer data storage including data representing the selected subset of input features.
12 . The computer system of claim 11 , wherein the selected subset of input features represented by the second data structure further correspond to actions which can be performed for the given entity.
13 . The computer system of claim 12 , wherein the library further includes a set of actionable factors, wherein the library stores, for each of the actionable factors, a mapping between the actionable factor and one or more respective input features in the selected subset of input features.
14 . The computer system of claim 13 , wherein the library further includes, for each actionable factor in the set of actionable factors, human-understandable content describing actions which can be performed for the given entity for output by the graphical user interface.
15 . The computer system of claim 13 , wherein the actionable factors include medically modifiable factors.
16 . The computer system of claim 12 , wherein the given entity comprises a patient and the actions include specifying a treatment for the patient.
17 . The computer system of claim 12 , wherein the given entity comprises a patient and the actions include specifying a behavior change for the patient.
18 . The computer system of claim 1 , wherein the sensitivity analysis module performs a sensitivity analysis on the localized model for the given entity wherein the first set of input features are held constant and sensitivity of the output of the localized model to variations in data values for the second set of input features is determined.
19 . The computer system of claim 18 wherein the sensitivity analysis uses a linear regression model.
20 . The computer system of claim 18 wherein the sensitivity analysis uses a nonlinear regression model.
21 . The computer system of claim 18 wherein the sensitivity analysis uses a Bayesian model.
22 . The computer system of claim 13 , wherein the second data structure maps an input feature in the selected subset of input features to a respective data value for the input feature for use as the different data value for the input feature by the sensitivity analysis module.
23 . The computer system of claim 1 , wherein the second data structure maps an input feature in the selected subset of input features to a respective data value for the input feature for use as the different data value for the input feature by the sensitivity analysis module.
24 . The computer system of claim 1 , wherein the graphical user interface further configures the computer system to receive a data value for an input feature in the selected subset of input features for use as the different data value for that input feature by the sensitivity analysis module.
25 . The computer system of claim 24 , wherein, to receive the data value for the input feature, the graphical user interface configures the computer system to:
access current data values for the given patient for medically modifiable factors;
present the current data values in a display, and
receive inputs indicating changes to the current data values.
26 . The computer system of claim 25 , wherein the medically modifiable factors include one or more of weight of a patient, body mass index of the patient, physical activity level of the patient, diabetes control of the patient, or smoking habits of the patient.
27 . The computer system of claim 1 wherein the localized model is a linear model.
28 . A computer system, comprising:
a. a processing system comprising a processing device and computer storage, wherein the computer storage stores:
i. a first data structure including, for each entity in a plurality of entities, respective data values for the entity for input features of a trained computational model, wherein the trained computational model outputs a predicted outcome for an entity based on the respective data values for the entity for the input features, and
ii. a second data structure comprising data describing a set of actionable factors related to a selected subset of the input features, the selected subset having fewer features than the input features, the second data structure including, for each actionable factor:
a) respective human-understandable content describing an action which can be performed for the given entity to affect the actionable factor, and
b) a respective mapping between the actionable factor and one or more respective input features in the selected subset of input features; and
b. computer program code that, when processed by the processing system, configures the processing system to:
construct a localized model of the trained computational model for a given entity, wherein the localized model is equivalent to the trained computational model by having fixed values for input features based on the respective data values for the given entity from the first data structure for input features other than the selected subset of the input features identified by the second data structure, wherein the localized model has variable inputs corresponding to only the selected subset of input features identified by the second data structure and has an output providing a predicted outcome for the given entity,
generate a respective predicted outcome for the given entity for different data values by applying the different data values for the input features identified in the selected subset of the input features to the variable inputs of the localized model for the given entity, wherein the different data values are different from the data values for the selected subset of input features for the given entity as stored in the first data structure; and
c. a graphical user interface comprising computer program code that, when processed by the processing system, configures the processing system to:
access current data values for the given entity for input features related to the actionable factors,
apply the current data values to the input features of the trained computational model to cause the trained computational model to output a respective predicted outcome,
present the current data values and the respective predictive outcome from the trained computational model in a display,
receive an input indicating a change to at least one of the current data values to generate changed data values for at least one of the input features related to the actionable factors,
cause the processing system to generate a predicted outcome output from the localized model for the given entity based on the changed data values by applying the changed data values as the different data values to the input features of the localized model for the given entity, and
using the second data structure, output human-understandable content describing how the changed data values likely would affect the predicted outcome of the trained computational model as applied to the given entity based on the predicted outcome output from the localized model.