Automated malware family signature generation
The automatic generation of malware family signatures is disclosed. A set of metadata associated with a plurality of samples is received. The samples are clustered. For members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster is determined. The similarities are evaluated for suitability as a malware family signature. In the event the similarities are determined to be suitable as a malware family signature, a signature is generated.
1. A system, comprising:
a processor configured to:
receive a set of metadata associated with a plurality of samples;
cluster the samples;
determine, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster; and
evaluate the similarities for suitability as a malware family signature, including by generating a query encompassing the similarities and performing the query against a malware repository; and
a memory coupled to the processor and configured to provide the processor with instructions.
2. The system of claim 1 wherein the processor is further configured to determine, for a first sample included in the plurality of samples, a set of features comprising name-value pairs.
3. The system of claim 2 wherein determining the set of features includes performing a tokenization.
4. The system of claim 1 wherein the processor is further configured to assign weights to a set of tokens.
5. The system of claim 4 wherein the weights are assigned using term frequency-inverse document frequency analysis.
6. The system of claim 1 wherein the processor is further configured to generate a vector list that indicates, for a given sample, a set of tokens hit by the sample.
7. The system of claim 1 wherein clustering the samples includes performing multiple rounds of k-means clustering and selecting as output those clusters with consistent membership across the multiple rounds.
8. The system of claim 1 wherein determining the similarities includes determining a portion of metadata that is present in all members of the first cluster.
9. The system of claim 8 wherein evaluating the similarities includes comparing a size of the first cluster to a number of samples in a corpus that also includes the portion of metadata.
10. The system of claim 1 wherein evaluating the similarities further includes determining a quality score based at least in part on a number of members in the first cluster and a number of results provided in response to the query.
11. The system of claim 1 wherein the processor is further configured to iteratively perform the clustering, determining, and evaluating until a low quality threshold is reached.
12. The system of claim 11 wherein the processor is further configured to exclude metadata associated with samples for which malware signatures were assigned in a previous iteration, prior to performing a current iteration.
13. The system of claim 1 wherein the processor is further configured to generate the malware family signature.
14. The system of claim 1 wherein the processor is further configured to provide as output a list of malware samples matching the malware family signature.
15. A method, comprising:
receiving a set of metadata associated with a plurality of samples;
clustering the samples;
determining, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster; and
evaluating the similarities for suitability as a malware family signature, including by generating a query encompassing the similarities and performing the query against a malware repository.
16. The method of claim 15 further comprising generating a vector list that indicates, for a given sample, a set of tokens hit by the sample.
17. The method of claim 15 wherein clustering the samples includes performing multiple rounds of k-means clustering and selecting as output those clusters with consistent membership across the multiple rounds.
18. The method of claim 15 further comprising providing as output a list of malware samples matching the malware family signature.
19. The method of claim 15 further comprising determining, for a first sample included in the plurality of samples, a set of features comprising name-value pairs.
20. The method of claim 19 wherein determining the set of features includes performing a tokenization.
21. The method of claim 15 further comprising assigning weights to a set of tokens.
22. The method of claim 21 wherein the weights are assigned using term frequency-inverse document frequency analysis.
23. The method of claim 15 wherein determining the similarities includes determining a portion of metadata that is present in all members of the first cluster.
24. The method of claim 23 wherein evaluating the similarities includes comparing a size of the first cluster to a number of samples in a corpus that also includes the portion of metadata.
25. The method of claim 15 wherein evaluating the similarities further includes determining a quality score based at least in part on a number of members in the first cluster and a number of results provided in response to the query.
26. The method of claim 15 further comprising iteratively performing the clustering, determining, and evaluating until a low quality threshold is reached.
27. The method of claim 26 further comprising excluding metadata associated with samples for which malware signatures were assigned in a previous iteration, prior to performing a current iteration.
28. The method of claim 15 further comprising generating the malware family signature.
29. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving a set of metadata associated with a plurality of samples;
clustering the samples;
determining, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster; and
evaluating the similarities for suitability as a malware family signature, including by generating a query encompassing the similarities and performing the query against a malware repository.