Specific file detection baked into machine learning pipelines
A set of features including a first feature and a second feature is received at a server. A subset of the set of features is determined for use in generating a model usable by a device to locally make a malware classification decision. The device has reduced computing resources as compared to computing resources of the server. The subset of the set of features is used to generate the model. The generated model includes the first feature and does not include the second feature. A determination is made, at a time subsequent to the generation of the model, that an updated model should be deployed to the device. An updated model is generated.
1. A system, comprising:
a processor configured to:
receive, at a server, a set of features including a first feature and a second feature;
determine a subset of the set of features to use in generating a model usable by a device to locally make a malware classification decision, wherein the device has reduced computing resources as compared to computing resources of the server;
use the subset of the set of features to generate the model, wherein the generated model includes the first feature and wherein the generated model does not include the second feature, wherein the first feature includes a specific count of a number of times a specific n-gram appears in a given file, and wherein when a custom benign test file that includes the specific n-gram the specific count of times is encountered by the device, the device will classify the custom benign test file as malicious; and
determine, at a time subsequent to the generation of the model, that an updated model should be deployed to the device, and generate the updated model; and
a memory coupled to the processor and configured to provide the processor with instructions.
2. The system of claim 1 , wherein generating the model includes appending a constructed regression tree to the model.
3. The system of claim 2 , wherein the constructed regression tree classifies a given file based on a count of occurrences of a custom set of n-grams.
4. The system of claim 3 , wherein the custom set of n-grams is selected to classify benign test file as being malicious.
5. The system of claim 1 , wherein the processor is further configured to generate the custom benign file.
6. The system of claim 1 , wherein the custom benign test file includes a specific count of each n-gram included in the custom set of n-grams.
7. The system of claim 2 , wherein the processor is further configured to determine that the constructed regression tree does not return a non-zero value for any samples included in a corpus.
8. The system of claim 2 , wherein the appended constructed regression tree does not reduce accuracy of the model in detecting malicious files.
9. The system of claim 1 , wherein the set of features includes features extracted from a set of known malicious files.
10. The system of claim 1 , wherein the set of features includes features extracted from a set of known benign files.
11. The system of claim 1 , wherein the subset of the set of features is determined using mutual information.
12. The system of claim 1 , wherein the subset of the set of features is determined using Chi-squared score.
13. The system of claim 1 , wherein the determination that the updated model should be deployed is made in response to a false positive result reported by a data appliance.
14. A method, comprising:
receiving, at a server, a set of features including a first feature and a second feature;
determining a subset of the set of features to use in generating a model usable by a device to locally make a malware classification decision, wherein the device has reduced computing resources as compared to computing resources of the server;
using the subset of the set of features to generate the model, wherein the generated model includes the first feature and wherein the generated model does not include the second feature, wherein the first feature includes a specific count of a number of times a specific n-gram appears in a given file, and wherein when a custom benign test file that includes the specific n-gram the specific count of times is encountered by the device, the device will classify the custom benign test file as malicious; and
determining, at a time subsequent to the generation of the model, that an updated model should be deployed to the device, and generate the updated model.
15. The method of claim 14 , wherein generating the model includes appending a constructed regression tree to the model.
16. The method of claim 15 , wherein the constructed regression tree classifies a given file based on a count of occurrences of a custom set of n-grams.
17. The method of claim 16 , wherein the custom set of n-grams is selected to classify the custom benign test file as being malicious.
18. The method of claim 14 , wherein the processor is further configured to generate the custom benign file.
19. The method of claim 14 , wherein the custom benign test file includes a specific count of each n-gram included in the custom set of n-grams.
20. The method of claim 15 , further comprising determining that the constructed regression tree does not return a non-zero value for any samples included in a corpus.
21. The method of claim 15 , wherein the appended constructed regression tree does not reduce accuracy of the model in detecting malicious files.
22. The method of claim 14 , wherein the set of features includes features extracted from a set of known malicious files.
23. The method of claim 14 , wherein the set of features includes features extracted from a set of known benign files.
24. The method of claim 14 , wherein the subset of the set of features is determined using mutual information.
25. The method of claim 14 , wherein the subset of the set of features is determined using Chi-squared score.
26. The system of claim 14 , wherein the determination that the updated model should be deployed is made in response to a false positive result reported by a data appliance.
27. A computer program product embodied in a tangible computer readable storage medium and comprising computer instructions for:
receiving, at a server, a set of features including a first feature and a second feature;
determining a subset of the set of features to use in generating a model usable by a device to locally make a malware classification decision, wherein the device has reduced computing resources as compared to computing resources of the server;
using the subset of the set of features to generate the model, wherein the generated model includes the first feature and wherein the generated model does not include the second feature, wherein the first feature includes a specific count of a number of times a specific n-gram appears in a given file, and wherein when a custom benign test file that includes the specific n-gram the specific count of times is encountered by the device, the device will classify the custom benign test file as malicious; and
determining, at a time subsequent to the generation of the model, that an updated model should be deployed to the device, and generate the updated model.