Systems and methods for determining and detecting malware families
Disclosed are techniques for identifying a malware family to which a malware sample belongs. A method can include receiving, by a computer system, a collection of malware signature samples for a malware family, identifying a test malware sample for testing whether the collection of malware signature samples includes a malware sample that causes generation of at least one malware detection rule that does not satisfy rule quality criteria, injecting the test malware sample into the collection, simulating sequence generation of the collection based on: applying the at least one rule to the collection to cause the collection to generate malware signature sequences, and generating a count indicating a quantity of the generated malware signature sequences, then determining whether the count satisfies family inclusion criteria, and adding the test malware sample to a dictionary for the malware family based on determining the count satisfies the family inclusion criteria.
1 . A method for identifying a malware family to which a malware sample belongs, the method comprising:
receiving, by a computer system, a collection of malware signature samples for a malware family;
identifying, by the computer system, a test malware sample, wherein the test malware sample is used, by the computer system, to test whether the collection of malware signature samples includes a malware sample that causes generation of at least one malware detection rule that does not satisfy one or more rule quality criteria;
injecting, by the computer system, the test malware sample into the collection of malware signature samples;
simulating, by the computer system, sequence generation of the collection of malware signature samples based on an iterative, repeatable procedural process comprising:
applying the at least one malware detection rule to the collection of malware signature samples to cause the collection of malware signature samples to generate a set of malware signature sequences characteristic of the collection of malware signature samples including the injected test malware sample,
determining a numerical count indicating a quantity of malware signature sequences within the generated set of malware signature sequences, wherein the numerical count serves as a metric reflecting an impact of the injected test malware sample on the characteristic malware signature sequences of the collection, and
determining, by the computer system, whether the numerical count, as the metric, satisfies one or more family inclusion criteria specifically defined by thresholds related to the numerical count, satisfaction of which validates membership of the test malware sample within the malware family;
adding, by the computer system, one or more of the malware signature sequences to a dictionary of malware signature sequences for the malware family based on a determination that the numerical count satisfies the one or more family inclusion criteria; and
returning, by the computer system, the dictionary for the malware family to be used for updating or generating the at least one malware detection rule for the malware family.
2 . The method of claim 1 , wherein identifying, by the computer system, a test malware sample comprises retrieving the test malware sample from the dictionary of malware samples for the malware family.
3 . The method of claim 1 , wherein injecting, by the computer system, the test malware sample into the collection of malware signature samples comprises:
identifying a subset of malware signature samples amongst the collection of malware signature samples; and
injecting the test malware sample into the subset of the collection of malware signature samples,
wherein the test malware sample is a malware sample in the collection of malware signature samples that was not included in the subset of the malware signature samples.
4 . The method of claim 1 , wherein determining, by the computer system, whether the numerical count satisfies one or more family inclusion criteria comprises determining that injecting the test malware sample into the collection causes the count to be within a threshold range of an initial count of malware signature sequences generated by the collection, wherein the initial count was generated, by the computer system, before the test malware sample was injected into the collection.
5 . The method of claim 1 , wherein determining, by the computer system, whether the numerical count, as the metric, satisfies one or more family inclusion criteria specifically defined by thresholds related to the numerical count, satisfaction of which validates membership of the test malware sample within the malware family comprises determining that injecting the test malware sample into the collection causes the numerical count to be greater than a threshold count.
6 . The method of claim 1 , wherein determining, by the computer system, whether the numerical count, as the metric, satisfies one or more family inclusion criteria specifically defined by thresholds related to the numerical count, satisfaction of which validates membership of the test malware sample within the malware family comprises determining that injecting the test malware sample into the collection causes a similar or same type of malware signature sequences to be generated as a type of malware signature sequences generated by the collection before the test malware sample was injected into the collection.
7 . The method of claim 1 , further comprising: removing, by the computer system, the test malware sample from the collection based on a determination that the count does not satisfy the one or more family inclusion criteria.
8 . The method of claim 1 , further comprising: generating, by the computer system, one or more malware detection rules for the malware family based at least in part on the test malware sample that was added to the dictionary for the malware family.
9 . The method of claim 1 , wherein returning, by the computer system, the dictionary for the malware family comprises transmitting the dictionary to a malware rule engine that is configured to perform at least one of: (i) updating the at least one malware detection rule, (ii) generating one or more malware detection rules for the malware family based on the dictionary for the malware family, or (iii) identifying malware instances in network traffic using the updated at least one malware detection rule or the generated one or more malware detection rules.
10 . The method of claim 1 , further comprising:
generating, by the computer system, a sub-family of malware signature samples from the collection of malware signature samples;
injecting, by the computer system, the test malware sample into the sub-family;
simulating, by the computer system, sequence generation of the sub-family that includes the test malware sample;
determining, by the computer system, whether the simulated sequence generation of the sub-family satisfies one or more sub-family inclusion criteria;
annotating, by the computer system, the test malware sample as being a member of the sub-family based on a determination that the one or more sub-family inclusion criteria is satisfied; and
adding, by the computer system, the annotated test malware sample to at least one of: the dictionary for the malware family or a dictionary for the sub-family.
11 . The method of claim 10 , wherein the sub-family includes malware signature samples from the collection that have at least one of a same: versioning, obfuscator, or portion of a byte sequence.
12 . The method of claim 1 , further comprising:
retrieving, by the computer system, a threshold-size set of malware signature samples for the malware family;
simulating, by the computer system, sequence generation of the threshold-size set of malware signature samples;
generating, by the computer system, a baseline count indicating a quantity of malware signature sequences that are generated during the simulating; and
comparing, by the computer system, the numerical count to the baseline count to determine whether the numerical count satisfies the one or more family inclusion criteria.
13 . The method of claim 12 , wherein the threshold-size set is one malware signature sample for the malware family.
14 . The method of claim 12 , further comprising:
injecting, by the computer system, the test malware sample into the threshold-size set of malware signature samples for the malware family; and
simulating, by the computer system, sequence generation of the threshold-size set of malware signature samples that includes the injected test malware sample.
15 . The method of claim 14 , further comprising:
injecting, by the computer system, another test malware sample into the threshold-size set of malware signature samples; and
simulating, by the computer system, sequence generation of the threshold-size set of malware signature samples that includes (i) the injected test malware sample and (ii) the another test malware sample.
16 . The method of claim 15 , further comprising: iteratively performing, by the computer system, the injecting and simulating steps until the threshold-size set of malware signature samples includes a total quantity of the malware signature samples for the malware family less one malware signature sample.
17 . The method of claim 1 , wherein the iterative, repeatable procedural process further comprises:
wildcarding, by the computer system, one or more of the malware signature sequences, by injecting a wildcard sequence into one or more of the malware signature sequences;
adding, by the computer system, the wildcarded malware signature sequences to the dictionary, such that the dictionary comprises both malware signature sequences and wildcarded malware signature sequences;
deconflicting the dictionary to remove conflicting malware signature sequences and wildcarded malware signature sequences from the dictionary; and
wherein returning, by the computer system, the dictionary for the malware family to be used for updating or generating the at least one malware detection rule for the malware family comprises returning, by the computer system, the deconflicted dictionary, comprising both malware signature sequences and wildcarded malware signature sequences, for the malware family to be used for updating or generating the at least one malware detection rule for the malware family.
18 . A system for identifying a malware family to which a malware sample belongs, the system comprising:
a computer system configured to generate malware detection rules for at least one malware family;
a data store configured to receive and store the malware detection rules generated by the computer system; and
a rule engine configured to retrieve the malware detection rules from the data store and detect, using the retrieved malware detection rules, malware instances in network traffic,
wherein the computer system comprises processors and memory and is configured to perform operations comprising:
retrieving, from the data store, a collection of malware signature samples for a malware family;
identifying a test malware sample, wherein the test malware sample is used, by the computer system, to test whether the collection of malware signature samples includes a malware sample that causes generation of at least one malware detection rule that does not satisfy one or more rule quality criteria;
injecting the test malware sample into the collection of malware signature samples;
simulating sequence generation of the collection of malware signature samples based on an iterative, repeatable procedural process comprising:
applying the at least one malware detection rule to the collection of malware signature samples to cause the collection of malware signature samples to generate a set of malware signature sequences characteristic of the collection of malware signature samples including the injected test malware sample,
determining a numerical count indicating a quantity of malware signature sequences within the generated set of malware signature sequences, wherein the numerical count serves as a metric reflecting an impact of the injected test malware sample on the characteristic malware signature sequences of the collection, and
determining whether the numerical count, as the metric, satisfies one or more family inclusion criteria specifically defined by thresholds related to the numerical count, satisfaction of which validates membership of the test malware sample within the malware family;
adding one or more of the malware to a dictionary of malware signature sequences for the malware family based on a determination that the numerical count satisfies the one or more family inclusion criteria; and
returning the dictionary for the malware family to be used for updating or generating the at least one malware detection rule for the malware family.
19 . The system of claim 18 , wherein:
the computer system is configured to perform operations comprising transmitting the deconflicted dictionary for the malware family to the rule engine, and
the rule engine is configured to perform operations comprising generating one or more malware detection rules for the malware family based at least in part on the deconflicted dictionary.
20 . The system of claim 19 , wherein the iterative, repeatable procedural process further comprises:
wildcarding, by the computer system, one or more of the malware signature sequences, by injecting a wildcard sequence into one or more of the malware signature sequences;
adding, by the computer system, the wildcarded malware signature sequences to the dictionary, such that the dictionary comprises both malware signature sequences and wildcarded malware signature sequences; and
deconflicting the dictionary to remove conflicting malware signature sequences and wildcarded malware signature sequences from the dictionary; and
wherein returning the dictionary for the malware family to be used for updating or generating the at least one malware detection rule for the malware family comprises returning the deconflicted dictionary, comprising both malware signature sequences and wildcarded malware signature sequences, for the malware family to be used for updating or generating the at least one malware detection rule for the malware family.