Uncertainty estimation using uninformative features
In some aspects, a computing system may generate uninformative features that may be added to a dataset of real features to use as a baseline for determining the quality of an explanation of model output. The uninformative features may be features that do not correlate with what a model is tasked with predicting (e.g., the uninformative features may be random values), and the real features may be informative and correlate with what the model is tasked with predicting (e.g., variables of a dataset sample). A machine learning model may be trained on a dataset that includes both the real features and the uninformative features. The computing system may generate feature attributions for model output, which may include feature attributions for the uninformative features and the real features in the dataset.
1 . A system for facilitating detection of uncertainty in machine learning model output through use of uninformative features, the system comprising:
one or more processors and one or more non-transitory media storing instructions that, when executed by the one or more processors, cause operations comprising:
accessing, via a network, a machine learning model, deployed in a production computing environment, that has been trained on a dataset comprising a set of uninformative features and a set of real features, wherein the set of uninformative features is generated such that the uninformative features are not correlated with correct labels of samples in the dataset, wherein the set of uninformative features is combined with the set of real features to form a combined set of features;
inputting, into the machine learning model, a modified sample that is derived from a sample for inference, wherein the sample comprises a first set of values corresponding to the set of real features and does not comprise a second set of values that corresponds to the set of uninformative features, wherein the modified sample comprises the first set of values and the second set of values; and
in connection with monitoring of the machine learning model deployed in the production computing environment, sending an electronic message, associated with the machine learning model, based on (i) a first local explanation indicating a ranking for each feature in the combined set of features and derived from the inputting of the modified sample into the machine learning model and (ii) more than a threshold number of uninformative features of the set of uninformative features being ranked higher than a first real feature of the set of real features, wherein the electronic message indicates an uncertainty level of the machine learning model.
2 . The system of claim 1 , the operations comprising sending, via a network, a modified explanation that is derived from the first local explanation and excludes the first real feature based on more than the threshold number of uninformative features being ranked higher than the first real feature.
3 . The system of claim 1 , wherein sending the electronic message associated with the machine learning model comprises not sending the electronic message until at least a determination that a set of explanations associated with a set of samples, comprising the sample, indicates that each sample in the set of samples comprises more than the threshold number of uninformative features that are ranked higher than a subset of the set of real features.
4 . The system of claim 1 , wherein sending the electronic message associated with the machine learning model comprises:
storing a first weighting of the first real feature and a second weighting of a second real feature, the first weighting being based on a first uninformative feature ranking higher than the first real feature, the second weighting being based on the first uninformative feature ranking lower than the second real feature; and
sending the electronic message based on the first weighting and the second weighting.
5 . A method comprising:
accessing, via a network, a machine learning model that has been trained on a dataset comprising a set of uninformative features and a set of real features, wherein the set of uninformative features is combined with the set of real features to form a combined set of features;
inputting, into the machine learning model, a modified sample derived from a sample for inference, wherein the sample comprises a first set of values corresponding to the set of real features, wherein the modified sample comprises the first set of values and a second set of values corresponding to the set of uninformative features; and
based on an indication of a ranking for one or more features in the combined set of features being derived from the inputting of the modified sample into the machine learning model, sending, via a network, a message associated with the machine learning model.
6 . The method of claim 5 , wherein a second ranking indicates that less than a threshold number of uninformative features are ranked higher than a first real feature, wherein the second ranking corresponds to a training sample of the dataset.
7 . The method of claim 5 , wherein sending the message associated with the machine learning model comprises sending the message based on a second variation derived from a batch of samples, comprising the modified sample, being different from a first variation associated with the dataset.
8 . The method of claim 5 , wherein sending the message associated with the machine learning model comprises sending the message based on a result indicating that more than a threshold number of uninformative features of the set of uninformative features is ranked higher than a first real feature of the set of real features.
9 . The method of claim 5 , wherein sending the message comprises sending the message comprising a given indication, derived from the ranking, that an uncertainty level associated with the machine learning model satisfies a threshold uncertainty level.
10 . The method of claim 5 , wherein sending the message comprises sending the message comprising a given indication, derived from the ranking, that a shift in distribution of samples satisfies a threshold distribution shift.
11 . The method of claim 5 , wherein sending the message associated with the machine learning model comprises sending the message based on a first weighting of a first real feature and a second weighting of a second real feature, the first weighting being based on a first uninformative feature ranking higher than the first real feature, the second weighting being based on the first uninformative feature ranking lower than the second real feature.
12 . The method of claim 5 , wherein sending the message associated with the machine learning model comprises sending the message based on a set of explanations associated with a set of samples, comprising the sample, indicating that each sample in the set of samples comprises more than a threshold number of uninformative features that are ranked higher than a subset of the set of real features.
13 . One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause operations comprising:
accessing a machine learning model, running on a computing device, that has been trained on a dataset comprising a set of uninformative features and a set of real features, wherein the set of uninformative features are combined with the set of real features to form a combined set of features;
inputting, into the machine learning model, a modified sample, comprising a first set of values corresponding to the set of real features and a second set of values corresponding to the set of uninformative features, to obtain an indication of a ranking for one or more features in the combined set of features; and
based on the ranking for the one or more features in the combined set of features, sending a message associated with the machine learning model.
14 . The one or more non-transitory computer-readable media of claim 13 , wherein a second ranking indicates that less than a threshold number of uninformative features are ranked higher than a first real feature, wherein the second ranking corresponds to a training sample of the dataset.
15 . The one or more non-transitory computer-readable media of claim 13 , wherein sending the message associated with the machine learning model comprises sending the message based on a second variation derived from a batch of samples, comprising the modified sample, being different from a first variation associated with the dataset.
16 . The one or more non-transitory computer-readable media of claim 13 , wherein sending the message associated with the machine learning model comprises sending the message based on a result indicating that more than a threshold number of uninformative features of the set of uninformative features is ranked higher than a first real feature of the set of real features.
17 . The one or more non-transitory computer-readable media of claim 13 , wherein sending the message comprises sending the message comprising a given indication, derived from the ranking, that an uncertainty level associated with the machine learning model satisfies a threshold uncertainty level.
18 . The one or more non-transitory computer-readable media of claim 13 , wherein sending the message comprises sending the message comprising a given indication, derived from the ranking, that a shift in distribution of samples satisfies a threshold distribution shift.
19 . The one or more non-transitory computer-readable media of claim 13 , wherein sending the message associated with the machine learning model comprises sending the message based on a first weighting of a first real feature and a second weighting of a second real feature, the first weighting being based on a first uninformative feature ranking higher than the first real feature, the second weighting being based on the first uninformative feature ranking lower than the second real feature.
20 . The one or more non-transitory computer-readable media of claim 13 , wherein sending the message associated with the machine learning model comprises sending the message based on a set of explanations associated with a set of samples indicating that each sample in the set of samples comprises more than a threshold number of uninformative features that are ranked higher than a subset of the set of real features.