System and method for network security
View Patent ↗A system for network security transparently occupies an observation port on the data stream, passing the entire range of network information to a dedicated interpreter. The interpreter resolves the data stream into individual data packets, which are then assembled into reconstructed network sessions according to parameters such as protocol type, source and destination addresses, source and destination ports, sequence numbers and other variables. The different types of sessions may include the traffic of many different types of users, such as e-mail, streaming video, voice-over-Internet and others. The system detects and stores the sessions into a database. A parser module may extract only the minimum information needed to reconstruct individual sessions. A backend interface permits a systems administrator to interrogate the forensic record of the network for maintenance, security and other purposes. The invention is not constrained to detect limited types of data, but rather captures and records a comprehensive record of network behavior.
1. A system for extracting information from network data, comprising:
an input interface connected to at least one source of network data; and
a network event sensor, communicating with the input interface, the network event sensor comprising
an interpreter module, the interpreter module scanning the network data to generate logical groupings of the network data, and
an assembler module, communicating with the interpreter module, the assembler module scanning the logical groupings to generate at least one session object,
wherein the network event sensor applies a lexical engine to the at least one session object recursively to identify protocols within other protocols to extract nested or underlying objects encapsulated in one or more different protocols and to identify at least one network event as at least one of a predetermined set of event types.
2. The system of claim 1 , wherein the at least one source of network data comprises an observation port connected to a network and continuously capturing network data from the network.
3. The system of claim 2 , wherein the observation port comprises a network interface card.
4. The system of claim 3 , wherein the network comprises at least one of an Ethernet network, a token ring network, and a TCP/IP network.
5. The system of claim 3 , wherein the network interface card is invisible to the network.
6. The system of claim 1 , wherein the at least one source of network data comprises stored network data.
7. The system of claim 6 , wherein the stored network data comprise at least one of captured network files, Website mirrors, archives of Usenet files, and archives of email files.
8. The system of claim 1 , wherein the logical groupings comprise packets.
9. The system of claim 1 , wherein the interpreter module removes low-level encoding information from the network data to generate the logical groupings.
10. The system of claim 9 , wherein the low-level encoding information removed by the interpreter module comprises hardware addressing information.
11. The system of claim 1 , wherein the at least one session object comprises at least one session file.
12. The system of claim 1 , wherein the assembler module scans the logical groupings by examining at least one of source address, destination address, sequence numbers, source port, and destination port to generate the at least one session object.
13. The system of claim 1 , wherein the lexical engine detects the presence of at least one predefined keyword to identify the at least one of a predetermined set of event types.
14. The system of claim 13 , wherein the predetermined set of event types comprises at least one of TCP, IP, UDP, SMTP, HTTP, NNTP, FTP, TELNET, DNS, RIP, BGP, MAIL, NEWS, HTML, XML, PGP, S/MIME, POP, IMAP, V-CARD, ICMP, NetBUI, IPX and SPX.
15. The system of claim 13 , wherein the lexical engine accumulates a total number of occurrences for the at least one predefined keyword to identify the event type.
16. The system of claim 15 , wherein the lexical engine applies a threshold to the number of occurrences to identify the event type.
17. The system of claim 1 , further comprising an extractor module, the extractor module extracting the at least one network event from the at least one session object according to the at least one of a predetermined set of event types.
18. The system of claim 17 , wherein the extractor module comprises a library of extractor types, each of the extractor types corresponding to at least one of the at least one of a predetermined set of event types.
19. The system of claim 18 , wherein the extractor module stores a minimum subset of the network data to reconstruct the at least one network event.
20. The system of claim 19 , wherein the minimum subset of the network data is stored in a database.
21. The system of claim 20 , further comprising a presentation module, communicating with the database, the presentation module querying the database for information related to the at least one network event.
22. The system of claim 1 , wherein the network event sensor also applies a port detection engine to the network data to identify the at least one network event.
23. The system of claim 1 , wherein the at least one source of network data comprises a plurality of sources of network data.
24. A method for extracting information from network data, comprising the steps of:
receiving network data from at least one source of network data;
scanning the network data to generate logical groupings of the network data;
scanning the logical groupings to generate at least one session object; and
recursively applying at least a lexical engine to the at least one session object to identify protocols within other protocols to extract nested or underlying objects encapsulated in one or more of the protocols and to identify more than one network event type contained in the at least one session object.
25. The method of claim 24 , wherein the at least one source of network data comprises an observation port connected to a network and continuously capturing network data from the network.
26. The method of claim 25 , wherein the observation port comprises a network interface card.
27. The method of claim 26 , wherein the network comprises at least one of an Ethernet network, a token ring network, and a TCP/IP network.
28. The method of claim 26 , wherein the network interface card is invisible to the network.
29. The method of claim 24 , wherein the at least one source of network data comprises stored network data.
30. The method of claim 29 , wherein the stored network data comprise at least one of captured network files, Website mirrors, archives of Usenet files, and archives of email files.
31. The method of claim 24 , wherein the logical groupings comprise packets.
32. The method of claim 24 , further comprising a step of removing low level encoding information from the network data to generate the logical groupings.
33. The method of claim 32 , wherein the low-level encoding information comprises hardware addressing information.
34. The method of claim 24 , wherein the at least one session object comprises at least one session file.
35. The method of claim 24 , wherein the step of scanning the logical groupings comprises a step of examining at least one of source address, destination address, sequence numbers, source port, and destination port to generate the at least one session object.
36. The method of claim 24 , further comprising a step of identifying the at least one network event as at least one of a predetermined set of event types.
37. The method of claim 36 , wherein the step of identifying comprises a step of detecting the presence of at least one predefined keyword to identify the at least one of a predetermined set of event types.
38. The method of claim 37 , wherein the predetermined set of event types comprises at least one of TCP, IP, UDP, SMTP, HTTP, NNTP, FTP, TELNET, DNS, RIP, BGP, MAIL, NEWS, HTML, XML, PGP, S/MIME, POP, IMAP, V-CARD, ICMP, NetBUI, IPX and SPX.
39. The method of claim 37 , wherein the step of detecting comprises a step of accumulating a total number of occurrences for the at least one predefined keyword to identify the event type.
40. The method of claim 39 , wherein the step of detecting comprises a step of applying a threshold to the number of occurrences to identify the event type.
41. The method of claim 36 , further comprising a step of extracting the at least one network event from the at least one session object according to the at least one of a predetermined set of event types.
42. The method of claim 41 , wherein the step of extracting comprises a step of selecting at least one extractor module from a library of extractor types, each of the extractor types corresponding to at least one of the at least one of a predetermined set of event types.
43. The method of claim 42 , further comprising a step of storing a minimum subset of the network data to reconstruct the at least one network event.
44. The method of claim 43 , wherein the step of storing comprises a step of storing the minimum subset of the network data in a database.
45. The method of claim 44 , further comprising a step of querying the database for information related to the at least one network event.
46. The method of claim 24 , further comprising a step of applying a port detection engine to the network data to identify the at least one network event.
47. The method of claim 24 , wherein the at least one source of network data comprises a plurality of sources of network data.