Network monitoring by using packet header analysis
View Patent ↗A computer method and a system for detecting the file type of an electronic file, the method including the steps of: (a) using a predetermined number of bytes at the beginning of the file to create a list of probable file types; (b) testing the file against a detection rule for each file type in the list until a match is found; if no match is found (c) testing the file against other known detection rules for the types to find a match.
1. A method for detecting a file type, the method comprising the steps of:
evaluating a predetermined number of bytes at a beginning of a file to create a list of probable file types;
testing the file against a detection rule for each file type in the list until a match is found; and
when no match is found, testing the file against other known detection rules for file types that are not included in the list to find a match.
2. The method of claim 1 , wherein the act of evaluating a predetermined number of bytes at a beginning of a file to create a list of probable file types comprises passing the file through a type-trie data structure.
3. The method of claim 2 , wherein each node of the type-trie data structure comprises a first path for a child node and a second path for a wild-card child.
4. The method of claim 3 , wherein the file is passed through the first path prior to being passed through the second path.
5. The method of claim 1 , wherein the detection rule for each file type in the list comprises examining a structure of the file.
6. The method of claim 1 , further comprising setting the file type to “unknown” if no match is found after testing the file against the other known detection rules for the file types that are not included in the list.
7. The method of claim 1 , further comprising, when no match is found after testing the file against the other known detection rules for the file types that are not included in the list, checking the file for a file extension and marking the file type as “unknown” when a recognizable type of file extension is found and when no file extension is found.
8. The method of claim 7 , further comprising, when an unrecognizable type of file extension is found, setting the file type corresponding to that unrecognizable file extension.
9. The method of claim 1 , wherein the detection rules are compiled into an intermediate state such that the detection rules can be quickly interpreted for testing.
10. A non-transitory computer-readable medium, comprising instructions stored thereon to cause one or more processing devices to:
evaluate a predetermined number of bytes at a beginning of a file to create a list of probable file types;
test the file against a detection rule for each file type in the list until a match is found; and
when no match is found, test the file against other known detection rules for file types that are not included in the list to find a match.
11. The non-transitory computer readable medium of claim 10 , wherein the act of evaluating a predetermined number of bytes at a beginning of a file to create a list of probable file types comprises passing the file through a type-trie data structure.
12. The non-transitory computer readable medium of claim 11 , wherein each node of the type-trie data structure comprises a first path for a child node and a second path for a wild-card child.
13. The non-transitory computer readable medium of claim 12 , wherein the file is passed through the first path prior to being passed through the second path.
14. The non-transitory computer readable medium of claim 10 , wherein the detection rule for each file type in the list comprises examining a structure of the file.
15. The non-transitory computer readable medium of claim 10 , further comprising, when no match is found after testing the file against the other known detection rules for the file types that are not included in the list, checking the file for a file extension and marking the file type as “unknown” when a recognizable type of file extension is found and when no file extension is found.
16. The non-transitory computer readable medium of claim 15 , further comprising, when an unrecognizable type of file extension is found, setting the file type corresponding to that unrecognizable file extension.
17. A computer system, comprising:
a memory;
one or more network adapters; and
a processing device communicatively coupled to the memory and configured to execute instructions stored in the memory to cause the processing device to:
capture data from network traffic communicated via the one or more network adapters;
evaluate a predetermined number of bytes at a beginning of the data to create a list of probable file types;
test the data against a detection rule for each file type in the list to find a match; and
when no match is found, test the data against other known detection rules for file types that are not included in the list to find a match.
18. The computer system of claim 17 , wherein the instructions to cause the processing device to evaluate a predetermined number of bytes at a beginning of the data to create a list of probable file types comprise instructions to cause the processing device to pass the data through a type-trie data structure.
19. The computer system of claim 18 , wherein each node of the type-trie data structure comprises a first path for a child node and a second path for a wild-card child.
20. The computer system of claim 18 , wherein the data is passed through the first path prior to being passed through the second path.