Detecting malware with hash-based fingerprints

1. A computer-implemented method for detecting malware using a hash-based fingerprint for an executable program, the computer-implemented method comprising the steps of:

executing the program on a computer system;

monitoring an event stream for the program, the event stream comprising a plurality of events;

calculating a first fingerprint, the first fingerprint calculated to represent a summary of the plurality of events in the event stream, wherein the first fingerprint comprises a plurality of hash values;

in response to an occurrence of a trigger event, comparing the first fingerprint with one or more second fingerprints, wherein the trigger event occurs when one or more hash values of the plurality of hash values comprising the first fingerprint include at least one of a trigger value and a trigger pattern, and wherein the trigger value and the trigger pattern comprises a lowest four bits of at least one hash value of the plurality of hash values all being zero; and

determining whether a behavior represented by the first fingerprint is a permitted behavior based, at least in part, on the comparison.

2. The computer-implemented method of claim 1 further comprising the step of:

recalculating a new value for the first fingerprint in response to a new event being added to the event stream, the new value for the first fingerprint recalculated based, at least in part, on a current value of the first fingerprint and a plurality of hash values associated with the new event, wherein the first fingerprint comprises a fixed length fingerprint, wherein the fixed length is irrespective of a number of events in the event stream.

3. The computer-implemented method of claim 1 , wherein the plurality of hash values are determined according to a MinHash hashing algorithm.

4. The computer-implemented method of claim 1 , wherein the plurality of events comprises API (Application Program Interface) calls.

5. The computer-implemented method of claim 1 , wherein the one or more second fingerprints represent one or more behaviors associated with known malware.

6. The computer-implemented method of claim 1 , wherein the step of calculating the first fingerprint includes grouping events in the event stream into a plurality of overlapping groups, and wherein the first fingerprint is calculated based, at least in part, on a group of events.

7. A non-transitory computer readable storage medium having a program stored thereon, the program causing the computer to execute the steps of:

executing an executable program on a computing device;

observing an event stream for the executable program, the event stream comprising a plurality of events;

generating a first fingerprint to represent a summary of the plurality of events in the event stream, wherein the first fingerprint comprises a plurality of hash values;

comparing the first fingerprint with one or more second fingerprints in response to a trigger event, wherein the trigger event occurs when one or more hash values of the plurality of hash values comprising the first fingerprint include at least one of a trigger value and a trigger pattern, and wherein the trigger value and the trigger pattern comprises a lowest four bits of at least one hash value of the plurality of hash values all being zero; and

determining, based at least in part on the comparison, whether a behavior represented by the first fingerprint is a permitted behavior.

8. The non-transitory computer readable storage medium of claim 7 , wherein the program further causes the computer to execute the step of:

regenerating a new value for the first fingerprint in response to a new event being added to the event stream, the new value for the first fingerprint regenerated based, at least in part, on a current value of the first fingerprint and a plurality of hash values associated with the new event, wherein the first fingerprint comprises a fixed length fingerprint wherein the fixed length is irrespective of a number of events in the event stream.

9. The non-transitory computer readable storage medium of claim 7 , wherein the plurality of hash values are determined according to a MinHash hashing algorithm.

10. The non-transitory computer readable storage medium of claim 7 , wherein the plurality of events comprises API (Application Program Interface) calls.

11. The non-transitory computer readable storage medium of claim 7 , wherein the one or more second fingerprints are associated with one or more programs known to contain malware.

12. The non-transitory computer readable storage medium of claim 7 , wherein the step of generating the first fingerprint includes grouping events in the event stream into a plurality of overlapping groups, and wherein the first fingerprint is calculated based on a group of events.

13. A system for detecting malicious software using a hash-based fingerprint, the system comprising:

a computing device to execute an executable program;

a monitoring module configured to monitor an event stream for the program, the event stream comprising a plurality of events;

an aggregating module configured to receive and process the plurality of events to generate a first fingerprint for the event stream, the first fingerprint generated to represent a summary of the plurality of events, wherein the first fingerprint comprises a plurality of hash values, wherein in response to a trigger event, the aggregating module provides the first fingerprint to a behavior analyzing module, wherein the trigger event occurs when one or more hash values of the plurality of hash values comprising the first fingerprint include at least one of a trigger value and a trigger pattern, and wherein the trigger value and the trigger pattern comprises a lowest four bits of at least one hash value of the plurality of hash values all being zero; and

the behavior analyzing module configured to receive the first fingerprint for the event stream, compare the first fingerprint for the event stream with one or more second fingerprints, and determine, based on the comparison, whether a behavior represented by the first fingerprint is a permitted behavior.

14. The system of claim 13 , wherein the aggregating module regenerates a new value for the first fingerprint in response to a new event being added to the event stream, the new value for the first fingerprint regenerated based, at least in part, on a current value of the first fingerprint and a plurality of hash values associated with the new event, wherein the first fingerprint comprises a fixed length fingerprint, wherein the fixed length is irrespective of a number of events in the event stream.

15. The system of claim 13 , wherein the plurality of hash values are determined according to a MinHash hashing algorithm.

16. The system of claim 13 , wherein the plurality of events comprises API (Application Program Interface) calls.

17. The system of claim 13 , wherein the one or more second fingerprints are associated with one or more programs known to contain malware.

18. The system of claim 13 , wherein the aggregating module is further configured to group the plurality of events in the event stream into a plurality of overlapping groups, and wherein the first fingerprint is generated based, at least in part, on a group of events.

19. A computer-implemented method for detecting malware using a hash-based fingerprint for an executable program, the computer-implemented method comprising the steps of:

executing the program on a computer system;

monitoring an event stream for the program, the event stream comprising a plurality of events;

calculating a first fingerprint, the first fingerprint calculated to represent a summary of the plurality of events in the event stream, wherein the first fingerprint comprises a plurality of hash values, wherein the plurality of hash values are determined according to a bMinHash hashing algorithm, and wherein a frequency of the trigger value and the trigger pattern is tunable by selecting a frequency feature of the bMinHash hashing algorithm;

in response to an occurrence of a trigger event, comparing the first fingerprint with one or more second fingerprints, wherein the trigger event occurs when one or more hash values of the plurality of hash values comprising the first fingerprint include at least one of a trigger value and a trigger pattern; and

determining whether a behavior represented by the first fingerprint is a permitted behavior based, at least in part, on the comparison.

20. The computer-implemented method of claim 19 further comprising the step of: recalculating a new value for the first fingerprint in response to a new event being added to the event stream, the new value for the first fingerprint recalculated based, at least in part, on a current value of the first fingerprint and a plurality of hash values associated with the new event, wherein the first fingerprint comprises a fixed length fingerprint, wherein the fixed length is irrespective of a number of events in the event stream.

21. The computer-implemented method of claim 19 , wherein the plurality of events comprises API (Application Program Interface) calls.

22. The computer-implemented method of claim 19 , wherein the one or more second fingerprints represent one or more behaviors associated with known malware.

23. The computer-implemented method of claim 19 , wherein the step of calculating the first fingerprint includes grouping events in the event stream into a plurality of overlapping groups, and wherein the first fingerprint is calculated based, at least in part, on a group of events.

24. A non-transitory computer readable storage medium having a program stored thereon, the program causing the computer to execute the steps of:

executing an executable program on a computing device;

observing an event stream for the executable program, the event stream comprising a plurality of events;

generating a first fingerprint to represent a summary of the plurality of events in the event stream, wherein the first fingerprint comprises a plurality of hash values, wherein the plurality of hash values are determined according to a bMinHash hashing algorithm, and wherein a frequency of the trigger value and the trigger pattern is tunable by selecting a frequency feature of the bMinHash hashing algorithm;

comparing the first fingerprint with one or more second fingerprints in response to a trigger event, wherein the trigger event occurs when one or more hash values of the plurality of hash values comprising the first fingerprint include at least one of a trigger value and a trigger pattern; and

determining, based at least in part on the comparison, whether a behavior represented by the first fingerprint is a permitted behavior.

25. The non-transitory computer readable storage medium of claim 24 , wherein the program further causes the computer to execute the step of:

regenerating a new value for the first fingerprint in response to a new event being added to the event stream, the new value for the first fingerprint regenerated based, at least in part, on a current value of the first fingerprint and a plurality of hash values associated with the new event, wherein the first fingerprint comprises a fixed length fingerprint wherein the fixed length is irrespective of a number of events in the event stream.

26. The non-transitory computer readable storage medium of claim 24 , wherein the plurality of events comprises API (Application Program Interface) calls.

27. The non-transitory computer readable storage medium of claim 24 , wherein the one or more second fingerprints are associated with one or more programs known to contain malware.

28. The non-transitory computer readable storage medium of claim 24 , wherein the step of generating the first fingerprint includes grouping events in the event stream into a plurality of overlapping groups, and wherein the first fingerprint is calculated based on a group of events.

29. A system for detecting malicious software using a hash-based fingerprint, the system comprising:

a computing device to execute an executable program;

a monitoring module configured to monitor an event stream for the program, the event stream comprising a plurality of events;

an aggregating module configured to receive and process the plurality of events to generate a first fingerprint for the event stream, the first fingerprint generated to represent a summary of the plurality of events, wherein the first fingerprint comprises a plurality of hash values, wherein the plurality of hash values are determined according to a bMinHash hashing algorithm, wherein a frequency of the trigger value and the trigger pattern is tunable by selecting a frequency feature of the bMinHash hashing algorithm, wherein in response to a trigger event, the aggregating module provides the first fingerprint to a behavior analyzing module, and wherein the trigger event occurs when one or more hash values of the plurality of hash values comprising the first fingerprint include at least one of a trigger value and a trigger pattern; and

the behavior analyzing module configured to receive the first fingerprint for the event stream, compare the first fingerprint for the event stream with one or more second fingerprints, and determine, based on the comparison, whether a behavior represented by the first fingerprint is a permitted behavior.

30. The system of claim 29 , wherein the aggregating module regenerates a new value for the first fingerprint in response to a new event being added to the event stream, the new value for the first fingerprint regenerated based, at least in part, on a current value of the first fingerprint and a plurality of hash values associated with the new event, wherein the first fingerprint comprises a fixed length fingerprint, wherein the fixed length is irrespective of a number of events in the event stream.

31. The system of claim 29 , wherein the plurality of events comprises API (Application Program Interface) calls.

32. The system of claim 29 , wherein the one or more second fingerprints are associated with one or more programs known to contain malware.

33. The system of claim 29 , wherein the aggregating module is further configured to group the plurality of events in the event stream into a plurality of overlapping groups, and wherein the first fingerprint is generated based, at least in part, on a group of events.