Anomaly detection performance enhancement using gradient-based feature importance
Herein are machine learning techniques that adjust reconstruction loss of a reconstructive model, such as a principal component analysis (PCA), based on importances of features. In an embodiment having a reconstructive model that more or less accurately reconstructs its input, a computer measures, for each feature, a respective importance that is based on the reconstructive model. For example, importance may be based on grading samples that the reconstructive model correctly or incorrectly inferenced. For each feature during production inferencing, a respective original loss from the reconstructive model measures a difference between a value of the feature in an input and a reconstructed value of the feature generated by the reconstructive model. For each feature, the respective importance of the feature is applied to the respective original loss to generate a respective weighted loss, which compensates for concept drift. The weighted losses of the features of the input are collectively detected as anomalous or non-anomalous.
1 . A method comprising:
measuring, for each feature of a plurality of features, a respective importance that is based on a) a reconstructive model, b) a gradient of the feature, and c) at least one selected from the group consisting of: a count of true positives, a count of false positives, an exponential term, and a z-score;
wherein the gradients of the plurality of features are based on a first plurality of log entries of a console log or a server log;
measuring, for each feature of the plurality of features in a second plurality of log entries of the console log or the server log, a respective original loss from the reconstructive model;
generating, for each feature of the plurality of features, a respective weighted loss of a plurality of weighted losses, wherein the weighted loss is based on: the original loss of the feature and the importance of the feature;
detecting, based on the plurality of weighted losses, that an anomalous subset from the second plurality of log entries of the console log or the server log are anomalous;
recording, for each log entry from the anomalous subset, a respective label that indicates a true positive or a false positive;
replacing, in the first plurality of log entries of the console log or the server log, a subset of the first plurality of log entries of the console log or the server log with the anomalous subset;
measuring, based on said replacing, recalculated importances of the plurality of features;
generating, based on the recalculated importances of the plurality of features and a plurality of original losses for a new log entry, a new plurality of weighted losses; and
detecting, based on the new plurality of weighted losses, that the new log entry is anomalous;
wherein the method is performed by one or more computers after the reconstructive model was trained.
2 . The method of claim 1 wherein said measuring the importance of the feature comprises measuring a plurality of gradients for the feature.
3 . The method of claim 2 wherein the plurality of gradients for the feature is based on the reconstructive model.
4 . The method of claim 2 wherein:
the method further comprises the reconstructive model generating a respective inference for each tuple of a plurality of tuples;
each gradient of the plurality of gradients for the feature is based on a respective tuple of the plurality of tuples.
5 . The method of claim 1 wherein the respective importance is further based on an eigenvector.
6 . The method of claim 5 wherein the respective importance is further based on at least one selected from the group consisting of:
a loss gradient,
a gradient for the feature of the importance,
a backpropogation, and
a sum or average of original losses.
7 . The method of claim 1 further comprising measuring, for each feature of a plurality of features, a respective second importance that is based on said labels of the log entries from the anomalous subset that indicate a true positive or a false positive.
8 . The method of claim 1 further comprising measuring, for each feature of a plurality of features, a respective second importance that is based on said plurality of weighted losses.
9 . The method of claim 1 wherein said reconstructive model is one selected from the group consisting of a principal component analysis (PCA), a recurrent neural network (RNN), a variational autoencoder (VAE), and an incremental PCA (IPCA).
10 . One or more computer-readable non-transitory media storing instructions that, when executed by one or more processors, cause after a reconstructive model was trained:
measuring, for each feature of a plurality of features, a respective importance that is based on a) the reconstructive model, b) a gradient of the feature, and c) at least one selected from the group consisting of: a count of true positives, a count of false positives, an exponential term, and a z-score;
wherein the gradients of the plurality of features are based on a first plurality of log entries of a console log or a server log;
measuring, for each feature of the plurality of features in a second plurality of log entries of the console log or the server log, a respective original loss from the reconstructive model;
generating, for each feature of the plurality of features, a respective weighted loss of a plurality of weighted losses, wherein the weighted loss is based on: the original loss of the feature and the importance of the feature;
detecting, based on the plurality of weighted losses, that an anomalous subset from the second plurality of log entries of the console log or the server log are anomalous;
recording, for each log entry from the anomalous subset, a respective label that indicates a true positive or a false positive;
replacing, in the first plurality of log entries of the console log or the server log, a subset of the first plurality of log entries of the console log or the server log with the anomalous subset;
measuring, based on said replacing, recalculated importances of the plurality of features;
generating, based on the recalculated importances of the plurality of features and a plurality of original losses for a new log entry, a new plurality of weighted losses; and
detecting, based on the new plurality of weighted losses, that the new log entry is anomalous.
11 . The one or more computer-readable non-transitory media of claim 10 wherein said measuring the importance of the feature comprises measuring a plurality of gradients for the feature.
12 . The one or more computer-readable non-transitory media of claim 11 wherein the plurality of gradients for the feature is based on the reconstructive model.
13 . The one or more computer-readable non-transitory media of claim 11 wherein:
the instructions further cause the reconstructive model generating a respective inference for each tuple of a plurality of tuples;
each gradient of the plurality of gradients for the feature is based on a respective tuple of the plurality of tuples.
14 . The one or more computer-readable non-transitory media of claim 10 wherein the respective importance is further based on an eigenvector.
15 . The one or more computer-readable non-transitory media of claim 14 wherein the respective importance is further based on at least one selected from the group consisting of:
a loss gradient,
a gradient for the feature of the importance,
a backpropogation, and
a sum or average of original losses.
16 . The one or more computer-readable non-transitory media of claim 10 wherein the instructions further cause measuring, for each feature of a plurality of features, a respective second importance that is based on said labels of the log entries from the anomalous subset that indicate a true positive or a false positive.
17 . The one or more computer-readable non-transitory media of claim 10 wherein the instructions further cause measuring, for each feature of a plurality of features, a respective second importance that is based on said plurality of weighted losses.
18 . The one or more computer-readable non-transitory media of claim 10 wherein said reconstructive model is one selected from the group consisting of a principal component analysis (PCA), a recurrent neural network (RNN), a variational autoencoder (VAE), and an incremental PCA (IPCA).