Balancing feature distributions using an importance factor
Herein are machine learning techniques that adjust reconstruction loss of a reconstructive model such as an autoencoder based on importances of values of features. In an embodiment and before, during, or after training, the reconstructive model that more or less accurately reconstructs its input, a computer measures, for each distinct value of each feature, a respective importance that is not based on the reconstructive model. For example, importance may be based solely on a training corpus. For each feature during or after training, a respective original loss from the reconstructive model measures a difference between a value of the feature in an input and a reconstructed value of the feature generated by the reconstructive model. For each feature, the respective importance of the input value of the feature is applied to the respective original loss to generate a respective weighted loss. The weighted losses of the features of the input are collectively detected as anomalous or non-anomalous.
1 . A method comprising:
measuring without using a reconstructive model:
a) for a distinct first categorical value of a particular feature of a plurality of features, a distinct first importance and
b) for a distinct second categorical value of the particular feature of the plurality of features, a distinct second importance; and
performing by a network switch:
i) measuring, for a network packet, a loss from the reconstructive model for the particular feature of the plurality of features;
ii) increasing the loss of the particular feature based on the second importance for the second categorical value of the particular feature;
iii) detecting, based on said increasing, that the network packet is anomalous; and
iv) rejecting, in response to said detecting, the network packet.
2 . The method of claim 1 wherein said measuring the first importance is based on an inverse document frequency that is based solely on categorical values of the particular feature.
3 . The method of claim 2 wherein:
the particular feature is one-hot encoded;
the inverse document frequency comprises a ratio having a denominator that is based solely on a count of the first categorical value of the particular feature.
4 . The method of claim 3 wherein each of the denominator and said count of the first categorical value of the particular feature is a count of the first categorical value of the particular feature in a training corpus.
5 . The method of claim 3 wherein:
a training corpus comprises a plurality of tuples;
each tuple in the plurality of tuples contains the plurality of features;
the ratio has a numerator that is not based on tuples in the plurality of tuples that lack a categorical value of the particular feature.
6 . The method of claim 1 further comprising manually assigning:
a first predefined multiplier to the first categorical value of the particular feature and
a second predefined multiplier to the second categorical value of the particular feature.
7 . The method of claim 1 further comprising at least one selected from a group consisting of:
training the reconstructive model based on the first importance,
training the reconstructive model before said measuring the first importance, and
training the reconstructive model after said measuring the first importance.
8 . The method of claim 1 wherein one is selected from a group consisting of:
said increasing occurs while training the reconstructive model and after training the reconstructive model,
said increasing occurs while training the reconstructive model but not after training the reconstructive model, and
said increasing occurs after training the reconstructive model but not while training the reconstructive model.
9 . The method of claim 1 wherein said increasing comprises a loss function accepting the first importance without the second importance.
10 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:
measuring without using a reconstructive model:
a) for a distinct first categorical value of a particular feature of a plurality of features, a distinct first importance and
b) for a distinct second categorical value of the particular feature of the plurality of features, a distinct second importance;
performing by a network switch:
i) measuring, for a network packet, a loss from the reconstructive model for the particular feature of the plurality of features;
ii) increasing the loss of the particular feature based on the second importance for the second categorical value of the particular feature;
iii) detecting, based on said increasing, that the network packet is anomalous; and
iv) rejecting, in response to said detecting, the network packet.
11 . The one or more non-transitory computer-readable media of claim 10 wherein said measuring the first importance is based on an inverse document frequency that is based solely on categorical values of the particular feature.
12 . The one or more non-transitory computer-readable media of claim 11 wherein:
the particular feature is one-hot encoded;
the inverse document frequency comprises a ratio having a denominator that is based solely on a count of the first categorical value of the particular feature.
13 . The one or more non-transitory computer-readable media of claim 12 wherein each of the denominator and said count of the first categorical value of the particular feature is a count of the first categorical value of the particular feature in a training corpus.
14 . The one or more non-transitory computer-readable media of claim 12 wherein:
a training corpus comprises a plurality of tuples;
each tuple in the plurality of tuples contains the plurality of features;
the ratio has a numerator that is not based on tuples in the plurality of tuples that lack a categorical value of the particular feature.
15 . The one or more non-transitory computer-readable media of claim 10 wherein the instructions further cause at least one selected from a group consisting of:
training the reconstructive model based on the first importance,
training the reconstructive model before said measuring the first importance, and
training the reconstructive model after said measuring the first importance.
16 . The one or more non-transitory computer-readable media of claim 10 wherein one is selected from a group consisting of:
said increasing occurs while training the reconstructive model and after training the reconstructive model,
said increasing occurs while training the reconstructive model but not after training the reconstructive model, and
said increasing occurs after training the reconstructive model but not while training the reconstructive model.
17 . The one or more non-transitory computer-readable media of claim 10 wherein said increasing comprises a loss function accepting the first importance without the second importance.