System and method for defining and detecting pestware
A system and method for defining and detecting pestware is described. One embodiment includes receiving a file and placing at least a portion of the file into a processor-readable memory of a computer. A plurality of execution paths within code of the pestware file are followed and particular instructions within the execution paths are identified. A representation of the relative locations of each of the particular instructions within the code of the file are compared against a pestware-definition file so as to determine whether the file is a potential pestware file.
1. A method for generating pestware definitions comprising:
receiving a pestware file;
placing at least a portion of the pestware file into a processor-readable memory;
following a plurality of execution paths within code of the pestware file, wherein each of the execution paths is a potential path that a processor executing the code will potentially follow;
identifying particular instructions within the execution paths;
storing, in a processor-readable pestware-definition file, a representation of the relative locations of each of the particular instructions within the code of the pestware file; and
sending the pestware-definition file to a plurality of client devices, wherein at least one of the client devices receives a file and determines whether relative locations of each of particular instructions within code of the file received by the at least one client device match at least a predetermined percentage of relative locations of instructions in the pestware-definition file.
2. The method of claim 1 , wherein the identifying includes identifying system calls within the execution paths.
3. The method of claim 1 including:
simplifying the representation of the relative locations so as to create a simplified representation of the relative locations, wherein the storing includes storing the representation as the simplified representation.
4. The method of claim 1 including:
bypassing, while following the plurality of execution paths, instructions other than system calls and jump instructions.
5. The method of claim 1 , wherein identifying particular instructions includes identifying calls to addresses to portions of the processor-readable memory that are outside of the memory occupied by the code of the pestware file.
6. The method of claim 5 , including:
storing a representation of each of the addresses in connection with a corresponding one of the locations so as to identify each call by an address.
7. The method of claim 6 , including:
storing a representation of each of the names of the calls in connection with a corresponding one of the locations so as to identify each call by a name.
8. A method for detecting pestware on a computer comprising:
receiving a file;
placing at least a portion of the file into a processor-readable memory of the computer;
following a plurality of execution paths within code of the file, wherein each of the execution paths is a potential path that a processor executing the code will potentially follow;
identifying particular instructions within the execution paths;
comparing, against a pestware-definition file, a representation of the relative locations of each of the particular instructions within the code of the file so as to determine whether the file is a potential pestware file; and
determining whether the relative locations of each of the particular instructions within the code match at least a predetermined percentage of relative locations of instructions in the pestware-definition file.
9. The method of claim 8 , wherein the identifying includes identifying system calls within the execution paths.
10. The method of claim 8 including:
simplifying the representation of the relative locations so as to create a simplified representation of relative locations, wherein the comparing includes comparing the representation as the simplified representation.
11. The method of claim 8 including:
bypassing, while following the plurality of execution paths, instructions other than system calls and jump instructions.
12. The method of claim 8 , wherein identifying particular instructions includes identifying calls to addresses to portions of the processor-readable memory that are outside of the memory occupied by the code of the pestware file.
13. The method of claim 8 including:
alerting a user of the computer in the event the relative locations of each of the particular instructions within the code match at least a predetermined percentage of relative locations of instructions in the pestware-definition file.
14. The method of claim 8 including:
quarantining the file in the event the relative locations of each of the particular instructions within the code match at least a predetermined percentage of relative locations of instructions in the pestware definition file.