Method and apparatus for detecting malicious software using machine learning techniques
Novel methods, components, and systems for detecting malicious software in a proactive manner are presented. More specifically, we describe methods, components, and systems that leverage machine learning techniques to detect malicious software. The disclosed invention provides a significant improvement with regard to detection capabilities compared to previous approaches.
1. One or more non-transitory computer readable storage media encoded with instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform operations comprising:
before execution of a software application, extracting a feature vector from the software application by applying a mathematical transformation operation to the software application to generate a series of values that represents features of the software application and that is indicative of whether or not the software application is likely to be benign or malicious; and
generating information indicative of a maliciousness of the software application by applying said feature vector to a classification algorithm concerning whether said software application is benign or potentially malicious, wherein the classification algorithm produces a score that represents a confidence in its determination as to whether the software application is benign or malicious.
2. The non-transitory computer readable storage media of claim 1 , further comprising instructions operable to perform operations including:
causing transmission of said feature vector to a server application that performs the generating information indicative of a maliciousness of the software application.
3. The non-transitory computer readable storage media of claim 1 , further comprising instructions operable to perform operations including:
accessing in a training phase a body of training data including a set of software applications; and
deriving during said training phase the classification algorithm for determining whether software applications are likely benign or malicious based on the training data.
4. The non-transitory computer readable storage media of claim 1 , wherein the data used to derive the classification algorithm is based on transaction logs of actual client systems.
5. The non-transitory computer readable storage media of claim 1 , wherein the classification algorithm is developed using a machine learning method including one or more of: Support Vector Machines, Neural Networks, Decision Trees, naive Bayes, or Logistic Regression.
6. The non-transitory computer readable storage media of claim 1 , further comprising instructions operable for comparing output of the classification algorithm to results from one or more other processes for identifying malicious software.
7. The non-transitory computer readable storage media of claim 1 , further comprising instructions operable for determining how to treat the software application based on results of the classification algorithm.
8. One or more non-transitory computer readable storage media encoded with instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform operations comprising:
accessing in a training phase a body of training data including a set of software applications to derive during said training phase a classification algorithm for determining whether selected software applications are likely benign or malicious;
receiving a feature vector relating to a software application of interest, wherein the feature vector is generated by applying a mathematical transformation operation to the software application of interest and the feature vector includes a series of values that represents one or more features of the software application of interest indicative of whether or not the software application of interest is likely to be benign or malicious;
applying the feature vector to the classification algorithm; and
generating information indicative of a maliciousness of the software application of interest based on results of the application of the feature vector to the classification algorithm.
9. The non-transitory computer readable storage media of claim 8 , wherein the feature vector is received from a client application, and further comprising instructions that, when executed by the one or more computer processors, are operable to transmit to the client application the information indicative of the maliciousness of the software application of interest.
10. The non-transitory computer readable storage media of claim 8 , wherein the classification algorithm produces a score that represents confidence in its determination as to whether the software application is benign or malicious.
11. The non-transitory computer readable storage media of claim 8 , wherein the data used to derive the classification algorithm is taken directly from transaction logs of actual client systems.
12. The non-transitory computer readable storage media of claim 8 , wherein the classification algorithm is developed using a machine learning method including one or more of: Support Vector Machines, Neural Networks, Decision Trees, naive Bayes, or Logistic Regression.
13. The non-transitory computer readable storage media of claim 8 , wherein the feature vector is encoded or compressed.
14. The non-transitory computer readable storage media of claim 8 , wherein the body of training data includes actual in-field usage data.
15. The non-transitory computer readable storage media of claim 8 , further comprising instructions operable for comparing output of the classification algorithm to results from another algorithm used to identify malicious software.
16. An apparatus comprising:
one or more network interfaces configured to transmit and receive data on a computer network;
a processor coupled to the network interfaces and configured to execute one or more processes; and
a memory configured to store instructions executable by the processor, when executed causing the processor to perform operations comprising:
accessing in a training phase a body of training data including a set of software applications to derive during said training phase a classification algorithm for determining
whether selected software applications are likely benign or malicious;
receiving a feature vector relating to a software application of interest, wherein the feature vector is generated by applying a mathematical transformation operation to the software application of interest and the feature vector includes a series of values that represents one or more features of the software application of interest indicative of whether or not the software application of interest is likely to be benign or malicious;
applying the feature vector to the classification algorithm; and
generating information indicative of a maliciousness of the software application of interest based on results of the application of the feature vector to the classification algorithm.
17. The apparatus of claim 16 , wherein the feature vector is received from a client application, and wherein the processor is operable to cause the information indicative of the maliciousness of the software application of interest to be transmitted, via the network interfaces, to the client application.
18. The apparatus of claim 16 , wherein the classification algorithm produces a score that represents confidence in its determination as to whether the software application of interest is benign or malicious.
19. The apparatus of claim 16 , wherein application of the classification algorithm is restricted to files whose popularity is below a specified threshold such that the software application of interest is flagged as being malicious if fewer than a predetermined number of systems appear to use the software application of interest.
20. The apparatus of claim 16 , wherein the feature vector includes information indicating one or more of: which dynamic linked libraries are referenced by the application, values of specific positions of binary contents, number of sections in a file and any attributes associated with those sections, number of symbols, size of a binary file, whether the binary file is digitally signed, a file system timestamp, properties of a filename, other installed software applications, or recent infection history.