Systems and methods for improving prediction process using automated rule learning framework
The disclosure relates to a method for receiving labelled data from data sources; generating a plurality of segments from the labelled data using tokenizers, wherein each of the segments is associated with one or more classes, and wherein the plurality of segments comprises a sub-plurality of unique segments; calculating an entropy value for each of the sub-plurality of unique segments; for each class, generating one or more rules based on each of the segments associated with the class and the entropy value of each such segment; and combining the generated one or more rules with a model to improve prediction performance of the model. The labelled data is input data for a classifier model to form predictions and false positives are extracted from the predictions. The model explainability techniques and the false positives are used to create additional rule.
1 . A computer-implemented method for improving a machine learning model, the method comprising:
receiving labelled data from one or more data sources;
clustering the labelled data when an amount of the labelled data is above a threshold value;
generating a plurality of segments from the labelled data, wherein the plurality of segments is associated with a plurality of classes, and wherein the plurality of segments comprises unique segments;
calculating an entropy value for each of the unique segments;
for each class in the plurality of classes, performing rule learning to generate one or more rules based on each of the plurality of segments associated with the class and the entropy value calculated for each unique segment associated with the class, wherein generating the one or more rules comprises determining one or more logical operators for each rule based on how far each corresponding entropy value deviates from a predefined value; and
combining the generated one or more rules with the machine learning model trained by the labelled data to reduce false positives and improve recall of the machine learning model.
2 . The method of claim 1 , wherein clustering the labelled data comprises performing clustering of centroid samples on the labelled data if the labelled data is above the threshold value.
3 . The method of claim 2 , wherein the clustering of centroid samples includes clustering a defined number of nearest centroids samples.
4 . The method of claim 2 , wherein the threshold value depends on configurations of a user and one or more hardware limitations of a system for improving a prediction process.
5 . The method of claim 1 , further comprising performing one or more pre-processing steps on the labelled data.
6 . The method of claim 1 , wherein each of the unique segments has one label.
7 . The method of claim 1 , wherein the labelled data is input data for a classifier model to form one or more predictions.
8 . The method of claim 7 , wherein one or more false positives are extracted from the one or more predictions.
9 . The method of claim 8 , wherein one or more model explainability techniques and the extracted one or more false positives are used to create additional rules.
10 . The method of claim 1 , wherein the plurality of segments is generated using one or more tokenizers, and one or more model explainability techniques are used to determine which token used by the one or more tokenizers results in misclassification of the labelled data.
11 . The method of claim 1 , wherein the one or more rules are created by a semantic rule builder.
12 . A computer system for improving a machine learning model, the system comprising:
a memory; and
one or more processors coupled with the memory, wherein the one or more processors, when executed, perform operations comprising:
receiving labelled data from one or more data sources;
clustering the labelled data when an amount of the labelled data is above a threshold value;
generating a plurality of segments from the labelled data, wherein the plurality of segments is associated with a plurality of classes, and wherein the plurality of segments comprises unique segments;
calculating an entropy value for each of the unique segments;
for each class in the plurality of classes, performing rule learning to generate one or more rules based on each of the plurality of segments associated with the class and the entropy value calculated for each unique segment associated with the class, wherein generating the one or more rules comprises determining one or more logical operators for each rule based on how far each corresponding entropy value deviates from a predefined value; and
combining the generated one or more rules with the machine learning model trained by the labelled data to reduce false positives and improve recall of the machine learning model.
13 . The system of claim 12 , wherein clustering the labelled data comprises performing clustering of centroid samples on the labelled data if the labelled data is above the threshold value.
14 . The system of claim 13 , wherein the clustering of centroid samples includes clustering a defined number of nearest centroids samples.
15 . The system of claim 13 , wherein the threshold value depends on configurations of a user and one or more hardware limitations for improving a prediction process.
16 . The system of claim 12 , further comprising performing one or more pre-processing steps on the labelled data.
17 . The system of claim 12 , wherein each of the unique segments has one label.
18 . The system of claim 12 , wherein the labelled data is input data for a classifier model to form one or more predictions.
19 . The system of claim 18 , wherein one or more false positives are extracted from the one or more predictions.
20 . A non-transitory computer readable medium containing computer-readable instructions stored therein for causing a computer processor to perform operations comprising:
receiving labelled data from one or more data sources;
clustering the labelled data when an amount of the labelled data is above a threshold value;
generating a plurality of segments from the labelled data, wherein the plurality of segments is associated with a plurality of classes, and wherein the plurality of segments comprises unique segments;
calculating an entropy value for each of the unique segments;
for each class in the plurality of classes, performing rule learning to generate one or more rules based on each of the plurality of segments associated with the class and the entropy value calculated for each unique segment associated with the class, wherein generating the one or more rules comprises determining one or more logical operators for each rule based on how far each corresponding entropy value deviates from a predefined value; and
combining the generated one or more rules with a machine learning model trained by the labelled data to reduce false positives and improve recall of the machine learning model.