Method and system for network-based detecting of malware from behavioral clustering
A computerized system and method for performing behavioral clustering of malware samples, comprising: executing malware samples in a controlled computer environment for a predetermined time to obtain Hypertext Transfer Protocol. HTTP traffic; clustering the malware samples into at least one cluster based on network behavioral information from the HTTP traffic; and extracting, using the at least one processor, network signatures from the HTTP traffic information for each cluster, the network signatures being indicative of malware infection.
1. A computerized method for detecting at least one malicious Hypertext Transfer Protocol (HTTP) request based on behavioral clustering of malware samples, comprising:
collecting information about at least one HTTP request from malware samples in a controlled computer environment for a predetermined time;
clustering, the malware samples into at least one cluster based on structural similarities among a plurality of the HTTP requests, the structural similarities comprising similarities between: a request method, Uniform Resource Locator (URL) path, URL page name, a parameter name, and a parameter value;
applying a single-linkage hierarchical clustering algorithm and a Davies-Bouldin (DB) cluster validity index to create a plurality of clusters;
merging together two or more of the plurality of clusters into meta clusters based on at least one HTTP behavior of each of the two or more of the plurality of clusters;
extracting network signatures from the information about the at least one HTTP request for each at least one cluster, the network signatures being indicative of malware infection; and
detecting at least one malicious HTTP request based on at least one of the extracted network signatures.
2. The method of claim 1 , further comprising clustering the malware samples into coarse-grain clusters based on statistical features extracted from the malicious HTTP request of the malware samples.
3. The method of claim 2 , further comprising finding a fine-grain distance between malware samples.
4. The method of claim 3 , further comprising applying a single-linkage hierarchical clustering algorithm and a Davies-Bouldin (DB) cluster validity index to split each coarse-grain cluster into fine-grain clusters based on structural features.
5. The method of claim 4 , wherein the merging together of the clusters comprises:
defining a cluster centroid for each of the fine-grain malware clusters;
defining distances between cluster centroids; and
grouping together malware samples that are very close to each other based on the determined distances.
6. The method of claim 5 , further comprising using the information about the at least one HTTP request generated by the malware samples in each meta-cluster as input to an automatic network signature generation algorithm.
7. The method of claim 6 , further comprising extracting network signatures from the malware samples in meta-clusters.
8. The method of claim 7 , further comprising filtering out network signatures that generate false positives.
9. A computerized system for detecting at least one malicious Hypertext Transfer Protocol (HTTP) request based on behavioral clustering of malware samples, comprising:
a non-transitory device comprising at least one processor configured for:
collecting information about at least one HTTP request information from malware samples in a controlled computer environment for a predetermined time;
clustering, using at least one processor, the malware samples into at least one cluster based on structural similarities among a plurality of the HTTP requests, the structural similarities comprising similarities between: a request method, Uniform Resource Locator (URL) path, URL page name, a parameter name, and a parameter value;
applying a single-linkage hierarchical clustering algorithm and a Davies-Bouldin (DB) cluster validity index to create a plurality of clusters;
merging together two or more of the plurality of clusters into meta clusters, based on at least one HTTP behavior or each of the two or more of the plurality of clusters;
extracting, using the at least one processor, network signatures from the information about the at least one HTTP request for each at least one cluster, the network signatures being indicative of malware infection; and
detecting, using the at least one processor, at least one malicious HTTP request based on at least one of the extracted network signatures.
10. The system of claim 9 , wherein the processor is further configured for clustering the malware samples into coarse-grain clusters based on statistical features extracted from the malicious HTTP request of the malware samples.
11. The system of claim 10 , wherein the processor is further configured for finding a fine-grain distance between malware samples.
12. The system of claim 11 , wherein the processor is further configured for applying a single-linkage hierarchical clustering algorithm and a Davies-Bouldin (DB) cluster validity index to split each coarse-grain cluster into fine-grain clusters based on structural features.
13. The system of claim 12 , wherein the merging together of the clusters comprises:
defining a cluster centroid for each of the fine-grain malware clusters;
defining distances between cluster centroids; and
grouping together malware samples that are very close to each other based on the determined distances.
14. The system of claim 13 , wherein the processor is further configured for using the information about the at least one HTTP request generated by the malware samples in each meta-cluster as input to an automatic network signature generation algorithm.
15. The system of claim 14 , wherein the processor is further configured for extracting network signatures from the malware samples in meta-clusters.
16. The system of claim 15 , wherein the processor is further configured for filtering out network signatures that generate false positives.
17. A computerized method for detecting at least one malicious Hypertext Transfer Protocol (HTTP) request based on behavioral clustering of malware samples, comprising:
collecting information about at least one HTTP request from malware samples in a controlled computer environment for a predetermined time;
clustering, the malware samples into at least one cluster based on statistical features among a plurality of HTTP requests, the statistical features comprising: a total number of HTTP requests, a number of GET requests, a number of POST requests, an average length of URLs, an average number of parameters in the request; an average amount of data sent by POST requests, and an average length of the response;
after clustering based on statistical features, clustering, the malware samples into at least one cluster based on structural similarities among the plurality of the HTTP requests;
identify a cluster centroid of each malware cluster;
identify a distance between cluster centroids;
merging together two or more of the plurality of clusters into meta clusters, based on the distance between the two or more cluster centroids;
extracting network signatures from the information about the at least one HTTP request for each at least one cluster, the network signatures being indicative of malware infection; and
detecting at least one malicious HTTP request based on at least one of the extracted network signatures.