Generating models for performing inline malware detection

1. A security platform for providing information to a data appliance that includes a firewall for blocking traffic determined to be malicious using the information, comprising:

a processor configured to:

receive a set of features, including a first plurality of n-grams, extracted from a set of files comprising a first plurality of known malicious files and a second plurality of known non-malicious files, wherein, for each file included in the set of files, respective histograms of n-grams present in the respective each file were previously generated during static analysis performed on the respective each file;

determine, using a mutual information technique and from the received set of features, a reduced set of features that includes a reduced set of the n-grams included in the first plurality of n-grams, wherein determining the reduced set of features includes preferentially including in the reduced set of features a first n-gram that is more indicative of maliciousness, because the first n-gram is present in the first plurality of known malicious files and is not present in the second plurality of known non-malicious files, than a second n-gram that is not preferentially included in the reduced set of features, because the second n-gram is present in the first plurality of known malicious files and is also present in the second plurality of known non-malicious files; and

use the reduced set of features to generate a model us able by the data appliance to perform in line malware analysis of samples received by the data appliance; wherein:

the generated model includes the first n-gram and the generated model does not include the second n-gram;

performing the inline malware analysis includes detecting as malicious a first sample, by the data appliance performing n-gram analysis on a sequence of packets of the first sample, received by the data appliance, wherein a maliciousness signature for the first sample is not present on the data appliance; and

using the first plurality of n-grams extracted from the set of known malicious and known non-malicious files in the inline malware analysis would render the data appliance inoperable;

and a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1 , wherein the reduced set of features includes features extracted from the first plurality of known malicious files.

3. The system of claim 1 , wherein the reduced set of features includes features extracted from the second plurality of known non-malicious files.

4. The system of claim 1 , wherein the generated model further includes at least one non n-gram feature.

5. The system of claim 4 , wherein the at least one non n-gram feature is associated with a file size.

6. The system of claim 4 , wherein the at least one non n-gram feature is associated with a header size.

7. The system of claim 4 , wherein the at least one non n-gram feature is associated with at least one of a presence or an absence of a checksum in a file.

8. The system of claim 4 , wherein the at least one non n-gram feature is associated with a number of sections in a file.

9. The system of claim 4 , wherein the at least one non n-gram feature is associated with a purported length of a file.

10. The system of claim 4 , wherein the at least one non n-gram feature is associated with whether a file includes an overlay.

11. The system of claim 1 , wherein the model is a linear model.

12. The system of claim 1 , wherein the model is a non-linear model.

13. The system of claim 1 , wherein in response to a false positive result reported by the data appliance, the processor is configured to generate an updated model and transmit the updated model to another data appliance.

14. A method for providing information to a data appliance that includes a firewall for blocking traffic determined to be malicious using the information, comprising:

receiving a set of features, including a first plurality of n-grams, extracted froma set of files comprising a first plurality of known malicious files and a second plurality of known non-malicious files, wherein, for each file included in the set of files, respective histograms of n-grams present in the respective each file were previously generated during static analysis performed on the respective each file;

determining, using a mutual information technique and from the received set of features, a reduced set of features that includes a reduced set of the n-grams included in the first plurality of n-grams, wherein determining the reduced set of features includes preferentially including in the reduced set of features a first n-gram that is more indicative of maliciousness, because the first n-gram is present in the first plurality of known malicious files and is not present in the second plurality of known non-malicious files, than a second n-gram that is not preferentially included in the reduced set of features, because the second n-gram is present in the first plurality of known malicious files and is also present in the second plurality of known non-malicious files; and

using the reduced set of features to generate a model us able by the data appliance to perform in line malware analysis of samples received by the data appliance; wherein:

the generated model includes the first n-gram and the generated model does not include the second n-gram;

performing the inline malware analysis includes detecting as malicious a first sample, by the data appliance performing n-gram analysis on a sequence of packets of the first sample, received by the data appliance, wherein a maliciousness signature for the first sample is not present on the data appliance; and

using the first plurality of n-grams extracted from the set of known malicious and known non-malicious files in the inline malware analysis would render the data appliance inoperable.

15. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving a set of features, including a first plurality of n-grams, extracted from a set of files comprising a first plurality of known malicious files and a second plurality of known non-malicious files, wherein, for each file included in the set of files, respective histograms of n-grams present in the respective each file were previously generated during static analysis performed on the respective each file;

determining, using a mutual information technique and from the received set of features, a reduced set of features that includes a reduced set of the n-grams included in the first plurality of n-grams, wherein determining the reduced set of features includes preferentially including in the reduced set of features a first n-gram that is more indicative of maliciousness, because the first n-gram is present in the first plurality of known malicious files and is not present in the second plurality of known non-malicious files, than a second n-gram that is not preferentially included in the reduced set of features, because the second n-gram is present in the first plurality of known malicious files and is also present in the second plurality of known non-malicious files; and

using the reduced set of features to generate a model us able by the data appliance to perform in line malware analysis of samples received by the data appliance; wherein:

the generated model includes the first n-gram and the generated model does not include the second n-gram;

performing the inline malware analysis includes detecting as malicious a first sample, by the data appliance performing n-gram analysis on a sequence of packets of the first sample, received by the data appliance, wherein a maliciousness signature for the first sample is not present on the data appliance; and

using the first plurality of n-grams extracted from the set of known malicious and known non-malicious files in the inline malware analysis would render the data appliance inoperable.

16. The method of claim 14 , wherein the reduced set of features includes features extracted from the first plurality of known malicious files.

17. The method of claim 14 , wherein the reduced set of features includes features extracted from the second plurality of known non-malicious files.

18. The method of claim 14 , wherein the generated model further includes at least one non n-gram feature.

19. The method of claim 18 , wherein the at least one non n-gram feature is associated with a file size.

20. The method of claim 18 , wherein the at least one non n-gram feature is associated with a header size.

21. The method of claim 18 , wherein the at least one non n-gram feature is associated with at least one of a presence or an absence of a checksum in a file.

22. The method of claim 18 , wherein the at least one non n-gram feature is associated with a number of sections in a file.

23. The method of claim 18 , wherein the at least one non n-gram feature is associated with a purported length of a file.

24. The method of claim 18 , wherein the at least one non n-gram feature is associated with whether a file includes an overlay.

25. The method of claim 14 , wherein the model is a linear model.

26. The method of claim 14 , wherein the model is a non-linear model.

27. The method of claim, 14 , further comprising, in response to receiving a false positive result reported by the data appliance, generating an updated model and transmit the updated model to another data appliance.