System and method for malware detection on a per packet basis

1. A computer implemented method for malware detection, the method comprising:

receiving a plurality of packets associated with a file, each of the plurality of packets comprising a content, and a source domain;

converting the content of a first packet of the plurality of packets into a decimal representation;

extracting one or more features from the converted content of the first packet, wherein the one or more features include one or more n-gram features, an entropy feature, and a domain feature;

applying a trained machine learning model to the extracted one or more features to determine a probability of maliciousness associated with the first packet;

responsive to determining that the probability maliciousness of the first packet is above a threshold value, labeling the file associated with the plurality of packets as malicious before having access to all of the plurality of packets that represent a complete file;

responsive to determining that the probability maliciousness of the first packet is below the threshold value, analyzing a next packet of the plurality of packets with the trained machine learning model to determine a second probability of maliciousness associated with the next packet; and

responsive to determining that the second probability of maliciousness of the next packet is above the threshold value, labeling the file as benign before having access to all of the plurality of packets that represent the complete file.

2. The computer implemented method of claim 1 , wherein the file is a one of a (portable executable) PE, a PDF file, and a Microsoft Office document.

3. The computer implemented method of claim 1 , wherein converting the content of the first packet comprises converting the packet into a digital representation.

4. The computer implemented method of claim 1 , wherein converting the content of the first packet comprises extracting a script associated with the file, the script including one of a JavaScript (JS), and a Visual Basic for Applications (VBA).

5. The computer implemented method of claim 4 , wherein converting the content of the first packet comprises tokenizing the extracted script.

6. The computer implemented method of claim 1 , wherein the one or more n-gram features is one of type n=1 and n=2.

7. The computer implemented method of claim 1 , wherein extracting one or more n-gram features from the converted packet further comprises tokenizing the converted packet.

8. The computer implemented method of claim 1 , wherein the trained machine learning model is one of a random forest and a random decision forest.

9. The computer implemented method of claim 1 further comprising training a machine learning model to determine a probability that a packet is malicious wherein training the machine learning model comprises:

retrieving one or more files from a malicious source and one or more file from a benign source;

receiving one or more selection criteria from a customer, the one or more selection criteria;

selecting one or more data based on the received one or more selection criteria;

generating one or more trees wherein each of the one or more trees comprises one or more nodes, one or more branches, and one or more termini associated with a decision boundary; and

applying bootstrap aggregation to train the one or more generated trees based on the selected one or more data.

10. The computer implemented method of claim 9 further comprising validating the trained machine learning model by:

receiving a target false positive rate from a customer;

determining a false positive rate associated with the trained machine learning model; and

responsive to the false positive rate being less than or equal to the target false positive rate, transmitting trained model to a malware detection engine.

11. The computer implemented method of claim 9 further comprising:

receiving a target false positive rate from a customer;

determining a false positive rate associated with the trained machine learning model; and

responsive to the false positive rate being greater than the target false positive rate, re-training the model.

12. The computer implemented method of claim 1 further comprising determining that the maliciousness of the file is uncertain when the determined prbability maliciousness of the first packet is below the threshold value and the determined second probability maliciousness of the next packet is above the second threshold value.

13. A system comprising:

a processor; and

a memory comprising stored instructions, the instructions executable by the processor and causing the processor to:

receive a plurality of packets associated with a file, each of the plurality of packets comprising a content, and a source domain;

convert the content of a first packet of the plurality of packets into a decimal representation;

extract one or more features from the converted content of the first packet, wherein the one or more features include one or more n-gram features, an entropy feature, and a domain feature;

apply a trained machine learning model to the extracted one or more features to determine a probability of maliciousness associated with the first packet;

responsive to determining that the probability of maliciousness of the first packet is above a threshold value, label the file associated with the plurality of packets as malicious before having access to all of the plurality of packets that represent a complete file;

responsive to determining that the probability maliciousness of the first packet is below the threshold value, analyze a next packet of the plurality of packets with the trained machine learning model to determine a second probability of maliciousness associated with the next packet, and

responsive to determining that the second probability of maliciousness of the next packet is above the threshold value, labeling the file as benign before having access to all of the plurality of packets that represent the complete file.

14. The system of claim 13 wherein the memory further comprises stored instructions executable by the processor cause the processor to:

retrieve one or more files from a malicious source and one or more file from a benign source;

receive one or more selection criteria from a customer, the one or more selection criteria;

select one or more data based on the received one or more selection criteria;

generate one or more trees wherein each of the one or more trees include one or more nodes, one or more branches, and one or more termini associated with a decision boundary; and

apply bootstrap aggregation to train the one or more generated trees based on the selected one or more data.

15. The system of claim 14 wherein the memory further comprises stored instructions executable by the processor that when executed cause the processor to:

receive a target false positive rate from the customer;

determine a false positive rate associated with the trained machine learning model; and

responsive to the false positive rate being less than or equal to the target false positive rate, transmit trained model to a malware detection engine.

16. The system of claim 14 wherein the memory further comprises stored instructions executable by the processor that when executed cause the processor to:

receive a target false positive rate from a customer;

determine a false positive rate associated with the trained machine learning model; and

responsive to the false positive rate being greater than the target false positive rate, re-train the model.

17. The system of claim 13 wherein the memory further comprises stored instructions executable by the processor that when executed cause the processor to:

determine that the maliciousness of the file is uncertain when the determined probability maliciousness of the first packet is below the threshold value and the determined second probability maliciousness of the next packet is above the second threshold value.

18. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions that when executed by a processor cause the processor to:

receive a plurality of packets associated with a file, each of the plurality of packets comprising a content, and a source domain;

convert the content of a first packet into a decimal representation;

extract one or more features from the converted content of the first packet, wherein the one or more features include one or more n-gram features, an entropy feature, and a domain feature;

apply a trained machine learning model to the extracted one or more features to determine a probability of maliciousness associated with the first packet;

responsive to determining that the probability maliciousness of the first packet is above a threshold value, label the file associated with the received packet as malicious before having access to all of the plurality of packets that represent a complete file;

responsive to determining that the probability of maliciousness of the first packet is below the threshold value, analyze a next packet of the plurality of packets with the trained machine learning model to determine a second probability of maliciousness associated with the next packet; and

responsive to determining that the second probability of maliciousness of the next packet is above the threshold value, labeling the file as benign before having access to all of the plurality of packets that represent the complete file.

19. The non-transitory computer readable medium of claim 18 further comprising instructions that when executed by the processor cause the processor to:

retrieve one or more files from a malicious source and one or more file from a benign source;

receive one or more selection criteria from a customer, the one or more selection criteria;

select one or more data based on the received one or more selection criteria;

generate one or more trees wherein each of the one or more trees include one or more nodes, one or more branches, and one or more termini associated with a decision boundary; and

apply bootstrap aggregation to train the one or more generated trees based on the selected one or more data.

20. The non-transitory computer readable medium of claim 19 further comprising instructions that when executed by the processor cause the processor to:

receive a target false positive rate from the customer;

determine a false positive rate associated with the trained machine learning model; and

responsive to the false positive rate being less than or equal to the target false positive rate, transmit trained model to a malware detection engine.