Identifying web shell applications through lexical analysis
Methods and systems for identifying malicious applications and, more particularly, to identifying web shell applications. Embodiments described herein rely on machine learning tools to analyze static and dynamic features of a suspected file to determine whether the file is a web shell application.
1. A method for identifying malicious applications, the method comprising:
accessing an application file through an interface to a computer-readable medium;
performing a lexical analysis on content of the application file, wherein the lexical analysis involves transforming the content of the application file into at least one individual letter or character;
determining a number of occurrences of the individual letter or character;
performing a calculation using the number of occurrences of the individual letter or character to determine at least one static feature related to the application file, wherein the at least one static feature comprises an entropy rate of the application file, and wherein the entropy rate is calculated based at least in part on a distribution of characters in the content of the application file;
providing the at least one static feature to a classifier executing a machine learning procedure, wherein the classifier is trained to analyze, for at least some application files: (i) static features, and (ii) dynamic features related to monitored behavior of the at least some application files when executed, wherein the dynamic features include attempted system calls or attempted network connections; and
receiving from the classifier a classification of the application file as a web shell.
2. The method of claim 1 wherein the at least one individual letter or character includes a syntactically meaningful token.
3. The method of claim 1 wherein the classifier is configured to execute a gradient boosting technique to compare the generated feature with a set of known web shells to classify the file as a web shell.
4. The method of claim 1 , further comprising determining whether the at least one individual letter or character includes a particular string or function call.
5. The method of claim 1 , wherein the at least one individual letter or character includes at least one individual function call.
6. The method of claim 1 , wherein the at least one static feature includes a number of times a particular character appears in a same place as in a known web shell.
7. The method of claim 1 wherein the generated feature comprises a string-comment ratio of the application file.
8. A system for identifying malicious applications, the system comprising:
an interface to a computer readable medium for accessing an application file;
a memory; and
a processor executing instructions stored on the memory to:
perform a lexical analysis on content of the application file, wherein the lexical analysis involves transforming the content of the application file into at least one individual letter or character,
determine a number of occurrences of the individual letter or character,
perform a calculation using the number of occurrences of the individual letter or character to generate a feature related to the file determine at least one static feature related to the application file, wherein the at least one static feature comprises an entropy rate of the application file, and wherein the entropy rate is calculated based at least in part on a distribution of characters in the content of the application file,
provide the at least one static feature to a classifier executing a machine learning procedure, wherein the classifier is trained to analyze, for at least some application files: (i) static features, and (ii) dynamic features related to monitored behavior of the at least some files when executed, wherein the dynamic features include attempted system calls or attempted network connections, and
receive from the classifier a classification of the application file as a web shell.
9. The system of claim 8 wherein the at least one individual letter or character includes a syntactically meaningful token.
10. The system of claim 8 wherein the classifier is configured to execute a gradient boosting technique to compare the at least one static feature with a set of known web shells to classify the application file as a web shell.
11. The system of claim 8 wherein the processor is further configured to determine whether the at least one individual letter or character includes a particular string or function call.
12. The system of claim 8 wherein the at least one individual letter or character includes at least one individual function call.
13. The system of claim 8 wherein the at least one static feature includes a number of times a particular character appears in a same place as in a known web shell.
14. The system of claim 8 wherein the generated feature is a string-comment ratio of the application file.