Machine learning classification with confidence threshold and similarity-based validation
View Patent ↗A method and a system for improving classification of data samples, which may be considered as class outliers, are claimed. The method includes inferring a pretrained classifying ML-based model on the incoming data sample, to assign a particular class of a plurality of classes thereto; calculating a similarity metric value representing a degree of similarity between the incoming data sample and one or more previously classified data samples of the particular class; and validating assignment of the particular class to the incoming data sample, based on the calculated similarity metric value.
1 . A method of classifying data samples by at least one processor, the method comprising:
receiving at least one incoming data sample;
inferring a pretrained classifying machine-learning (ML)-based model on the at least one incoming data sample, to assign a particular class of a plurality of classes thereto, and wherein said inferring further comprises:
calculating a confidence metric value representing confidence of assignment of the particular class to the at least one incoming data sample; and
triggering validating the assignment of the particular class to the at least one incoming data sample, when the calculated confidence metric value does not exceed a predefined confidence metric value threshold;
calculating a similarity metric value representing a degree of similarity between the at least one incoming data sample and one or more previously classified data samples of the particular class; and
validating assignment of the particular class to the at least one incoming data sample, based on the calculated similarity metric value, thereby obtaining at least one classified incoming data sample.
2 . The method of claim 1 , wherein calculating the similarity metric value is performed by using a validating ML-based model; wherein the validating ML-based model is configured to calculate the similarity metric value based on the at least one incoming data sample and the one or more previously classified data samples of the particular class.
3 . The method of claim 2 , wherein the validating ML-based model is a nearest neighbor type classifier.
4 . The method of claim 1 , wherein validating the assignment of the particular class to the at least one incoming data sample further comprises determining that the calculated similarity metric value exceeds a predefined similarity metric value threshold.
5 . The method of claim 4 , wherein calculating the similarity metric value with respect to the one or more previously classified data samples of the particular class is performed serially, to find at least one of the previously classified data samples, for which the calculated similarity metric value exceeds the predefined similarity metric value threshold.
6 . The method of claim 1 , wherein the at least one incoming data sample and the one or more previously classified data samples are represented as vectors and the similarity metric value is a cosine similarity metric value.
7 . The method of claim 1 , wherein the classifying ML-based model is pretrained so as to assign classes of the plurality of classes to data samples.
8 . The method of claim 7 , wherein the classifying ML-based model is pretrained based on an initial training dataset, the initial training dataset comprising a plurality of data samples having classes of the plurality of classes assigned thereto.
9 . The method of claim 8 , wherein the method further comprises performing supplementary training of the classifying ML-based model, based on the at least one classified incoming data sample.
10 . The method of claim 9 , wherein the supplementary training of the classifying ML-based model comprises
supplementing the initial training dataset with the at least one classified incoming data sample, thereby obtaining a supplemented training dataset; and
training the classifying ML-based model to assign classes of the plurality of classes to data samples, based on the supplemented training dataset.
11 . A system for classifying data samples, the system comprising: a non-transitory memory device, wherein modules of instruction code are stored, and at least one processor associated with the memory device, and configured to execute the modules of instruction code, whereupon execution of said modules of instruction code, the at least one processor is configured to:
receive at least one incoming data sample;
infer a pretrained classifying machine-learning (ML)-based model on the at least one incoming data sample, to assign a particular class of a plurality of classes thereto, and wherein said inferring further comprises:
calculating a confidence metric value representing confidence of assignment of the particular class to the at least one incoming data sample; and
triggering validating the assignment of the particular class to the at least one incoming data sample, when the calculated confidence metric value does not exceed a predefined confidence metric value threshold;
calculate a similarity metric value representing a degree of similarity between the at least one incoming data sample and one or more previously classified data samples of the particular class; and
validate assignment of the particular class to the at least one incoming data sample, based on the calculated similarity metric value, thereby obtaining at least one classified incoming data sample.
12 . The system of claim 11 , wherein the at least one processor is further configured to validate the assignment of the particular class to the at least one incoming data sample by determining that the calculated similarity metric value exceeds a predefined similarity metric value threshold.
13 . The system of claim 11 , wherein the at least one incoming data sample and the one or more previously classified data samples are represented as vectors and the similarity metric value is a cosine similarity metric value.
14 . The system of claim 11 , wherein the classifying ML-based model is pretrained so as to assign classes of the plurality of classes to data samples.
15 . The system of claim 14 , wherein the classifying ML-based model is pretrained based on an initial training dataset, the initial training dataset comprising a plurality of data samples having classes of the plurality of classes assigned thereto.
16 . The system of claim 15 , wherein the at least one processor is further configured to perform supplementary training of the classifying ML-based model, based on the at least one classified incoming data sample.
17 . The system of claim 16 , wherein the at least one processor is further configured to perform supplementary training of the classifying ML-based model by
supplementing the initial training dataset with the at least one classified incoming data sample, thereby obtaining a supplemented training dataset; and
training the classifying ML-based model to assign classes of the plurality of classes to data samples, based on the supplemented training dataset.
18 . The system of claim 11 , wherein the at least one processor is further configured to calculate the similarity metric value by using a validating ML-based model; wherein the validating ML-based model is configured to calculate the similarity metric value based on the at least one incoming data sample and the one or more previously classified data samples of the particular class; and wherein the validating ML-based model is based on a nearest neighbor type classifier.