Systems and methods for instructions-based detection of sophisticated obfuscation and packing
Disclosed herein are new methods and systems for detecting obfuscated programs. We build a recursive traversal disassembler that extracts the control flow graph of binary files. This allows us to detect the presence of interleaving instructions, which is typically an indication of the opaque predicate anti-disassembly trick. Our detection system uses some novel features based on referenced instructions and the extracted control flow graph that clearly distinguishes between obfuscated and normal files. When these are combined with a few features based on file structure, we achieve a very high detection rate of obfuscated files.
1. A system for instructions-based detection of sophisticated obfuscation comprising:
one or more hardware processors;
and a storage medium that stores computer program logic that is executable by the one or more hardware processors, said computer program logic comprising:
criteria to test if a file is packed; and
wherein said criteria is further comprised of checking if
the sink vertices ratio lies outside the range of nonpacked files.
2. A system for instructions-based detection of sophisticated obfuscation comprising:
one or more hardware processors;
and a storage medium that stores computer program logic that is executable by the one or more hardware processors, said computer program logic comprising:
criteria to test if a file is packed; and
wherein said criteria is further comprised of
checking if the sink vertices ratio lies outside the range of nonpacked files;
checking if the referenced instructions ratio lies outside the range of non-packed files;
checking if the average number of instructions in basic block lies outside the range of non-packed files;
checking if the entropy of referenced instructions lies outside the range on non-packed files;
checking if the code has one or more anti-disassembly tricks; and
checking if the code references an unknown opcode.
3. A system for instructions-based detection of sophisticated packing comprising:
one or more hardware processors;
and a storage medium that stores computer program logic that is executable by the one or more hardware processors, said computer program logic comprising:
criteria to identify obfuscated files; and
wherein said criteria is further comprised of checking if
SizeOfRawData=0 and VirtualSize>0 for some section.
4. A system for instructions-based detection of sophisticated packing comprising:
one or more hardware processors;
and a storage medium that stores computer program logic that is executable by the one or more hardware processors, said computer program logic comprising:
criteria to identify obfuscated files; and
wherein said criteria is further comprised of checking if
the sum of SizeOfRawData field of all sections is greater than the file size.
5. A system for instructions-based detection of sophisticated packing comprising:
one or more hardware processors;
and a storage medium that stores computer program logic that is executable by the one or more hardware processors, said computer program logic comprising:
criteria to identify obfuscated files; and
wherein said criteria is further comprised of checking if
the file has no imports at all or the import table is corrupted.
6. A system for instructions-based detection of sophisticated packing comprising:
one or more hardware processors;
and a storage medium that stores computer program logic that is executable by the one or more hardware processors, said computer program logic comprising:
criteria to identify obfuscated files; and
wherein said criteria is further comprised of
checking if the entry point is in file header before any section;
checking if there is no .text or CODE section in the file;
checking if the entry point is in the last section while it is neither .text nor CODE section;
checking if SizeOfRawData=0 and VirtualSize>0 for some section;
checking the sum of SizeOfRawData field of all sections is greater than the file size;
checking if two or more sections overlap; and
checking if the file has no imports at all or the import table is corrupted.
7. A system for instructions-based detection of sophisticated obfuscation and packing comprising:
one or more hardware processors;
and a storage medium that stores computer program logic that is executable by the one or more hardware processors, said computer program logic comprising:
criteria to test if a file is packed and identify obfuscated files;
wherein said criteria is comprised of
checking if the sink vertices ratio lies outside the range of nonpacked files;
checking if the referenced instructions ratio lies outside the range of non-packed files;
checking if the average number of instructions in basic block lies outside the range of non-packed files;
checking if the entropy of referenced instructions lies outside the range on non-packed files;
checking if the code has one or more anti-disassembly tricks;
checking if the code references an unknown opcode;
checking if the entry point is in file header before any section;
checking if there is no .text or CODE section in the file;
checking if the entry point is in the last section while it is neither .text nor CODE section;
checking if SizeOfRawData=0 and VirtualSize>0 for some section;
checking the sum of SizeOfRawData field of all sections is greater than the file size;
checking if two or more sections overlap; and
checking if the file has no imports at all or the import table is corrupted.