Automated malware family signature generation
The automatic generation of malware family signatures is disclosed. A set of metadata associated with a plurality of samples is received. The samples are clustered. For members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster is determined. The similarities are evaluated for suitability as a malware family signature. Suitability is evaluated based on how well the similarities uniquely identify the members of the first cluster. In the event the similarities are determined to be suitable as a malware family signature, a signature is generated.
1. A system, comprising:
a processor configured to:
receive a set of metadata associated with a plurality of samples;
cluster the samples;
determine, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster;
evaluate the similarities for suitability as a malware family signature, wherein suitability is evaluated based at least in part on how well the similarities uniquely identify the members of the first cluster; and
generate the malware family signature, including by generating one or more rules using at least a portion of the set of similarities and validating rule quality; and
a memory coupled to the processor and configured to provide the processor with instructions.
2. The system of claim 1 wherein the processor is further configured to determine, for a first sample included in the plurality of samples, a set of features comprising name-value pairs.
3. The system of claim 2 wherein determining the set of features includes performing a tokenization.
4. The system of claim 1 wherein the processor is further configured to assign weights to a set of tokens.
5. The system of claim 4 wherein the weights are assigned using term frequency-inverse document frequency analysis.
6. The system of claim 1 wherein the processor is further configured to generate a vector list that indicates, for a given sample, a set of tokens hit by the sample.
7. The system of claim 1 wherein clustering the samples includes performing multiple rounds of k-means clustering and selecting as output those clusters with consistent membership across the multiple rounds.
8. The system of claim 1 wherein determining the set of similarities includes determining a portion of metadata that is present in all members of the first cluster.
9. The system of claim 8 wherein evaluating the similarities includes comparing a size of the first cluster to a number of samples in a corpus that also includes the portion of metadata.
10. The system of claim 1 wherein the processor is further configured to iteratively perform the clustering, determining, and evaluating until a low quality threshold is reached.
11. The system of claim 10 wherein the processor is further configured to exclude metadata associated with samples for which malware signatures were assigned in a previous iteration, prior to performing a current iteration.
12. The system of claim 1 wherein the processor is further configured to provide as output a list of malware samples matching the malware family signature.
13. A method, comprising:
receiving a set of metadata associated with a plurality of samples;
clustering the samples;
determining, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster;
evaluating the similarities for suitability as a malware family signature, wherein suitability is evaluated based at least in part on how well the similarities uniquely identify the members of the first cluster; and
generating the malware family signature, including by generating one or more rules using at least a portion of the set of similarities and validating rule quality.
14. The method of claim 13 further comprising assigning weights to a set of tokens.
15. The method of claim 14 wherein the weights are assigned using term frequency-inverse document frequency analysis.
16. The method of claim 13 further comprising generating a vector list that indicates, for a given sample, a set of tokens hit by the sample.
17. The method of claim 13 wherein clustering the samples includes performing multiple rounds of k-means clustering and selecting as output those clusters with consistent membership across the multiple rounds.
18. The method of claim 13 further comprising providing as output a list of malware samples matching the malware family signature.
19. The method of claim 13 further comprising determining, for a first sample included in the plurality of samples, a set of features comprising name-value pairs.
20. The method of claim 19 wherein determining the set of features includes performing a tokenization.
21. The method of claim 13 wherein determining the set of similarities includes determining a portion of metadata that is present in all members of the first cluster.
22. The method of claim 21 wherein evaluating the similarities includes comparing a size of the first cluster to a number of samples in a corpus that also includes the portion of metadata.
23. The method of claim 13 further comprising iteratively performing the clustering, determining, and evaluating until a low quality threshold is reached.
24. The method of claim 23 further comprising excluding metadata associated with samples for which malware signatures were assigned in a previous iteration, prior to performing a current iteration.
25. A computer program product embodied in a computer readable storage memory and comprising computer instructions for:
receiving a set of metadata associated with a plurality of samples;
clustering the samples;
determining, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster;
evaluating the similarities for suitability as a malware family signature, wherein suitability is evaluated based at least in part on how well the similarities uniquely identify the members of the first cluster; and
generating the malware family signature, including by generating one or more rules using at least a portion of the set of similarities and validating rule quality.