Pattern similarity measures to quantify uncertainty in malware classification

1. A non-transitory computer-readable storage medium having computer-readable code stored thereon for programming one or more processors to perform steps of:

receiving content for classification;

prior to transforming features of the received content, performing an initial classification of the content, wherein the initial classification categorizes the content as one of benign and malicious utilizing a model that has been trained with a training set of data including benign data and malicious data;

selecting and transforming features of the received content;

determining a first pattern associated with the received content based on the transformed features, the content being one of an executable file, an office document, and a Portable Document File (PDF);

comparing the first pattern with a second pattern, wherein the second pattern is associated with one of the benign data or the malicious data from the training set of data based on the initial classification;

determining an uncertainty of the initial classification based on a distance between the first pattern and the second pattern; and

blocking or allowing the content based on the initial classification and the distance being above or below a threshold, wherein the content is blocked if the content is initially categorized as malicious and the distance between the first pattern and a second pattern associated with malicious data from the training set of data is less than a first threshold, and wherein the content is allowed if the content is initially categorized as benign and the distance between the first pattern and a second pattern associated with benign data from the training set of data is less than a second threshold.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the computer-readable code is further configured to program the one or more processors to perform steps of

discarding the classification if the distance is greater than a configurable threshold.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the computer-readable code is further configured to program the one or more processors to perform steps of

determining the model has drifted based on the distance and one or more previous distances from previously classified content; and

causing retraining of the model with a new training set of data, including new benign data and malicious data.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the distance represents a similarity between the received content and one of the benign data and the malicious data used to train model.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the first pattern and the second pattern are determined utilizing one of Principal Component Analysis (PCA) and Auto-encoder.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the first pattern and the second pattern are determined based on a plurality of features including raw byte ngram count and frequency, word frequency, entropy, and file size.

7. The non-transitory computer-readable storage medium of claim 1 , wherein the distance is determined as a reconstruction error mapping a pattern to its original feature space.

8. A system comprising:

a network interface;

a processor communicatively coupled to the network interface; and

memory storing computer-executable instructions that, when executed, cause the processor to

receive content for classification,

prior to transforming features of the received content, perform an initial classification of the content, wherein the initial classification categorizes the content as one of benign and malicious utilizing a model that has been trained with a training set of data including benign data and malicious data;

select and transform features of the received content;

determine a first pattern associated with the received content based on the transformed features, the content being one of an executable file, an office document, and a Portable Document File (PDF);

compare the first pattern with a second pattern, wherein the second pattern is associated with one of the benign data or the malicious data from the training set of data based on the initial classification;

determine an uncertainty of the initial classification based on a distance between the first pattern and the second pattern; and

block or allow the content based on the initial classification and the distance being above or below a threshold, wherein the content is blocked if the content is initially categorized as malicious and the distance between the first pattern and a second pattern associated with malicious data from the training set of data is less than a first threshold, and wherein the content is allowed if the content is initially categorized as benign and the distance between the first pattern and a second pattern associated with benign data from the training set of data is less than a second threshold.

9. The system of claim 8 , wherein the computer-executable instructions, when executed, further cause the processor to

discard the classification if the distance is greater than a configurable threshold.

10. The system of claim 8 , wherein the computer-executable instructions, when executed, further cause the processor to

determine the model has drifted based on the distance and one or more previous distances from previously classified content; and

cause retraining of the model with a new training set of data, including new benign data and malicious data.

11. The system of claim 8 , wherein the distance represents a similarity between the received content and one of the benign data and the malicious data used to train model.

12. The system of claim 8 , wherein the first pattern and the second pattern are determined utilizing one of Principal Component Analysis (PCA) and Auto-encoder.

13. The system of claim 8 , wherein the first pattern and the second pattern are determined based on a plurality of features including raw byte ngram count and frequency, word frequency, entropy, and file size.

14. A method comprising:

receiving content for classification;

prior to transforming features of the received content, performing an initial classification of the content, wherein the initial classification categorizes the content as one of benign and malicious utilizing a model that has been trained with a training set of data including benign data and malicious data;

selecting and transforming features of the received content;

determining a first pattern associated with the received content based on the transformed features, the content being one of an executable file, an office document, and a Portable Document File (PDF);

comparing the first pattern with a second pattern, wherein the second pattern is associated with one of the benign data or the malicious data from the training set of data based on the initial classification;

determining an uncertainty of the initial classification based on a distance between the first pattern and the second pattern; and

blocking or allowing the content based on the initial classification and the distance being above or below a threshold, wherein the content is blocked if the content is initially categorized as malicious and the distance between the first pattern and a second pattern associated with malicious data from the training set of data is less than a first threshold, and wherein the content is allowed if the content is initially categorized as benign and the distance between the first pattern and a second pattern associated with benign data from the training set of data is less than a second threshold.

15. The method of claim 14 , further comprising

discarding the classification if the distance is greater than a configurable threshold.

16. The method of claim 14 , further comprising

determining the model has drifted based on the distance and one or more previous distances from previously classified content; and

causing retraining of the model with a new training set of data including new benign data and malicious data.