Systems, apparatus, and methods for detecting malware
View Patent ↗Various embodiments, including a method comprising creating a first fuzzy fingerprint of a known malware file, the first fuzzy fingerprint including a first set of calculated complexity approximations and weightings for each of a plurality of blocks within the known malware file, creating a second fuzzy fingerprint of a file to be checked, the second fuzzy fingerprint including a second set of calculated complexity approximations and weightings for each of a plurality of blocks within the file to be checked, comparing the second fuzzy fingerprint to the first fuzzy fingerprint, calculating a similarity probability for each of the block-wise comparisons, the calculation including a respective weightings for each of the plurality of blocks within the known malware file and for each of the plurality of blocks within the file to be checked, and the calculation including a distance between the compared blocks; and calculating an overall similarity probability for the plurality of blocks compared.
1. A method comprising:
creating a first fuzzy fingerprint of a known malware file, the first fuzzy fingerprint including a first set of calculated complexity approximations and weightings for each of a plurality of blocks within the known malware file;
creating a second fuzzy fingerprint of a file to be checked, the second fuzzy fingerprint including a second set of calculated complexity approximations and weightings for each of a plurality of blocks within the file to be checked;
comparing the second fuzzy fingerprint to the first fuzzy fingerprint via a computerized system, the comparing including comparing the calculated complexity approximations from the second fuzzy fingerprint with a plurality of the complexity approximations from the first fuzzy fingerprint using a plurality of block-wise comparisons of the plurality of blocks within the known malware file and the plurality of blocks within the file to be checked; and
calculating a similarity probability for each of the block-wise comparisons, the calculation including a respective weighting for each of the plurality of blocks within the known malware file and for each of the plurality of blocks within the file to be checked, and the calculation including an actual deviation of both blocks complexity approximation in relation to a maximum possible deviation, times a proximity factor for the comparison between an offset i in a first file x 1 and an offset j in a second file x 2 ; and
calculating an overall similarity probability for the plurality of blocks compared, the overall similarity probability therefore comprising a similarity probability between the known malware file and the file to be checked.
2. The method of claim 1 , including stopping the block-wise comparisons if a threshold number of block-wise comparisons in a row each dropped below a threshold value N for the calculated similarity probability.
3. The method of claim 1 , calculating a complexity approximation for each of a plurality of blocks within the known malware file and calculating a complexity approximation for each of the plurality of blocks within the file to be checked includes calculating the information entropy for a given block using the formula:
H
(
X
)
=
-
∑
i
=
1
n
p
(
x
i
)
log
2
p
(
x
i
)
where
H(X) is based on the information content or self-information of X, which is itself an arbitrary variable;
p(x i )=Pr(X=x i ) is the probability mass function of X; and
is calculated with x={0..255}, x i =i−1 and n=|x|.
4. The method of claim 1 , including summing up using Bayes' formula to generate the overall similarity probability, wherein Bayes' formula relates the conditional and marginal probabilities of stochastic events A and B:
Pr
(
A
|
B
)
=
Pr
(
B
|
A
)
Pr
(
A
)
Pr
(
B
)
∝
L
(
A
|
B
)
Pr
(
A
)
where L(A|B) is the likelihood of A given fixed B.
5. The method of claim 1 , wherein calculating the overall similarity probability for the plurality of blocks compared includes comparing the overall similarity probability to a threshold value M to determine if the second file is to be considered malware.
6. The method of claim 5 , including determining that the file to be checked is a variant of the known malware file when the calculated overall similarity probability equals or exceeds the threshold value M.
7. The method of claim 1 , wherein calculating the overall similarity probability for the plurality of blocks compared includes comparing the overall similarity probability to a threshold value X and a threshold value M to determine if the second file is to be considered malware or is to be considered a suspicious file, wherein the threshold value M is greater than the threshold value X.
8. The method of claim 7 , wherein when the calculated overall similarity probability is less than the threshold value X the file to be checked is determined not to be a variant of the known malware file, and when the calculated overall similarity probability is greater than threshold value M the file to be checked is determined to be a variant of the known malware file, and the file to be checked is considered a suspicious file when the calculated overall similarity probability is greater than the threshold value X but less than or equal to the threshold value for M.
9. An apparatus comprising:
a gateway including an anti-malware engine coupled to a generated fuzzy fingerprints database including plurality of fuzzy fingerprints for known malware files, each fuzzy fingerprint for a known malware file including a first set of calculated complexity approximations and weightings for each of a plurality of blocks within the known malware file;
a fuzzy fingerprint generator coupled to the anti-malware engine, the fuzzy fingerprint generator operable to produce a fuzzy fingerprint including a complexity approximation for each of a plurality of blocks for a file to be checked provided by the anti-malware engine; and
a fingerprint comparator coupled to the anti-malware engine, the fingerprint comparator operable to compare a produced fingerprint from the fuzzy fingerprint generator with any one of the plurality of fuzzy fingerprints from the generated fuzzy fingerprints database by calculating a similarity probability for each of the block-wise comparisons, the calculation including respective weighting for each of the plurality of blocks within the known malware file and for each of the plurality of blocks within the file to be checked and the calculation including an actual deviation of both blocks complexity approximation in relation to a maximum possible deviation, times a proximity factor for the comparison between an offset i in a first file x i and an offset j in a second file x 2 , thereby producing a similarity probability between the known malware file and the file to be checked.
10. The apparatus of claim 9 , producing a complexity approximation for each of a plurality of blocks includes calculating the information entropy for a given block using the formula:
H
(
X
)
=
-
∑
i
=
1
n
p
(
x
i
)
log
2
p
(
x
i
)
where
H(X) is based on the information content or self-information of X which is itself an arbitrary variable;
p(x i )=Pr(X=x i ) is the probability mass function of X; and
is calculated with x={0..255}, x i =i−1 and n=|x|.
11. The apparatus of claim 9 , including summing up using Bayes' formula to generate an overall similarity probability, wherein Bayes' formula relates the conditional and marginal probabilities of stochastic events A and B:
Pr
(
A
|
B
)
=
Pr
(
B
|
A
)
Pr
(
A
)
Pr
(
B
)
∝
L
(
A
|
B
)
Pr
(
A
)
where L(A|B) is the likelihood of A given fixed B.
12. The apparatus of claim 9 , including an update mechanism operable to receive updated versions of generated fuzzy fingerprints for storage in the generated fuzzy fingerprints database.
13. The apparatus of claim 9 , wherein the generated fuzzy fingerprints database includes a precondition table that contains magic bytes signatures for all media type covered by the database.
14. The apparatus of claim 9 , wherein the generated fuzzy fingerprints database includes a jump table operable to determine a relative offset into the generated fuzzy fingerprints database based on a file size of the file to be checked.
15. A system comprising:
a plurality of protected devices coupled to a network through a gateway, the gateway comprising a computerized system including an anti-malware engine;
a generated fuzzy fingerprint database coupled to the anti-malware engine, the generated fingerprint database including plurality of fingerprints for known malware files coupled to the anti-malware engine, the fingerprint for each known malware file comprising a first set of calculated approximations and weightings for each of a plurality of blocks within the known malware file;
a fuzzy fingerprint generator coupled to the anti-malware engine, the fuzzy fingerprint generator operable to produce a fuzzy executable fingerprint including a complexity approximation for each of a plurality of blocks in a file provided by the anti-malware engine; and
a fuzzy fingerprint comparator coupled to the anti-malware engine, the fuzzy fingerprint comparator operable to compare a produced fuzzy executable fingerprint from the fingerprint generator for the file to be checked with any one of a plurality of fingerprints from the generated fingerprint database by calculating a similarity probability for each of the block-wise comparisons, the calculation including respective weighting for each of the plurality of blocks within the known malware file and for each of the plurality of blocks within the file to be checked, and the calculation including an actual deviation of both blocks complexity approximation in relation to a maximum possible deviation, times a proximity factor for the comparison between an offset i in a first file x 1 and an offset i in a second file x 2 , thereby producing a similarity probability between the known malware file and the file to be checked.
16. The system of claim 15 , wherein the generated fuzzy fingerprints database is coupled to an update mechanism, the update mechanism operable to receive an updated version of the generated fuzzy fingerprints database.
17. The system of claim 16 , wherein the updated version update is created from a training set of known malware files on a vendor-provided fingerprint generator server.
18. The system of claim 15 , wherein the generated fuzzy fingerprints database is coupled to an update mechanism, the update mechanism operable to receive a new version of the fuzzy executable fingerprints for known malware files.
19. The system of claim 18 , wherein the new version of fuzzy executable fingerprints for known malware files is an incremental update of a previous version of fuzzy executable fingerprints for known malware files.
20. The system of claim 15 , including an update provisioning server coupled to an update mechanism, the update provisioning server operable to couple the update provisioning server to a fuzzy fingerprint generator server.
21. The system of claim 15 , including a fuzzy fingerprint generator server operable to store a plurality of known malware files to be used as one or more training sets for generating a fuzzy executable fingerprint for each of a known malware file.