Connection table for intrusion detection
A system for detecting network intrusions and other conditions in a network is described. The system includes a plurality of collector devices that are disposed to collect data and statistical information on packets that are sent between nodes on a network. An aggregator device is disposed to receive data and statistical information from the plurality of collector devices. The aggregator device produces a connection table that maps each node on the network to a record that stores information about traffic to or from the node. The aggregator runs processes that determine network events from aggregating of anomalies into network events.
1. A computer system for tracking network behavior, comprising:
a processor;
a memory that stores:
a connection table that maps each host of a network to a record that stores traffic information from the host to other hosts or from the other hosts to the host in the network for a specified time interval, and
a profile table that stores historical traffic information as exponentially weighted moving average values; and
a merging mechanism configured to merge the record associated with each host for the specified time interval from the connection table into the historical traffic information in the profile table.
2. The computer system of claim 1 wherein the connection table includes a plurality of records that are indexed by source address.
3. The computer system of claim 1 wherein the connection table includes a plurality of records that are indexed by destination address.
4. The computer system of claim 1 wherein the connection table includes a plurality of records that are indexed by time.
5. The computer system of claim 1 wherein the connection table includes a plurality of records, that are record objects, which are indexed by source address, destination address and time.
6. The computer system of claim 1 wherein the connection table is a plurality of connection sub-tables each sub-table having data pertaining to network traffic over different time scales.
7. The computer system of claim 6 wherein the connection sub-tables include a time-slice connection table that operates on a small unit of time and at least one other sub-table that operates on a larger unit of time than the time slice sub-table.
8. The computer system of claim 7 wherein the at least one other sub-table holds records received from collectors over the time scale of the table.
9. The computer system of claim 5 wherein an address indexing the connection table comprises an IP address.
10. The computer system of claim 1 wherein an address indexing the connection table includes a physical layer address to IP address map that is used to determine Host ID.
11. The computer system of claim 1 wherein a host record of a first host maps that first host to a second host that communicates with the first host to a host pair record object that has information about traffic from the first to the second host and from the second host to the first host.
12. The computer system of claim 1 wherein the connection table includes two level mapping that enables a consuming device to obtain summary information about one host for a first level mapping and about traffic between any pair of hosts, in either direction, between a first host of the any pair to a second host of the any pair and from the second host of the any pair to the first host of the any pair for a second level mapping.
13. The computer system of claim 1 wherein the connection table comprises a plurality of host records, a host record stores a measure of the number of bytes, packets, and connections that occurred between hosts during a time-period.
14. The computer system of claim 13 , wherein data in the host record is organized by well known transport protocols and well-known application-level protocols.
15. The computer system of claim 13 , wherein host records have no specific memory limit.
16. The computer system of claim 1 wherein for application-level protocols and for every pair of hosts, the connection table stores statistics for traffic between hosts.
17. The computer system of claim 16 wherein the connection table stores protocol-specific records as (protocol, count) key-value pairs.
18. A computer system for tracking network behavior, the computer system comprising:
a processor;
a memory that stores:
a connection table that maps each host of a network to a record that stores traffic information from the host to other hosts or from the other hosts to the host in the network for a specified time interval,
wherein the connection table is indexed according to one or more of source address, destination address and a specified time interval, and
wherein the connection table includes records fields for storing statistical information for traffic between the hosts; and
a profile table that stores historical traffic information as exponentially weighted moving average values; and
a merging mechanism configured to merge the record associated with each host for the specified time interval from the connection table into the historical traffic information in the profile table.
19. The computer system of claim 18 wherein the plurality of records is record objects.
20. The computer system of claim 18 wherein the connection table is a second plurality of connection sub-tables, each sub-table having data pertaining to network traffic over different ones of corresponding second plurality of time scales.
21. The computer system of claim 18 wherein the connection sub-tables include a time-slice connection table that operates on a small unit of time and at least one other sub-table that operates on a larger unit of time than the time slice sub-table.
22. The computer system of claim 18 wherein the at least one other sub-table holds records received from collectors in the network over the time scale of the table.
23. The computer system of claim 18 wherein an address indexing the connection table comprises an IP address.
24. The computer system of claim 23 wherein an address indexing the connection table includes a physical layer address to IP address map that is used to determine Host ID.
25. The computer system of claim 18 wherein a host record of a first host maps that first host to a second host that communicates with the first host to a host pair record that has information about traffic from the first to the second host and from the second host to the first host.
26. The computer system of claim 18 wherein the connection table includes two level mapping that enables a consuming device to obtain summary information about one host for a first level mapping and about the traffic between any pair of hosts, in either direction, between a first host of the any pair to a second host of the any pair and from the second host of the any pair to the first host of the any pair for a second level mapping.
27. The computer system of claim 18 wherein the connection table comprises a plurality of host records, a host record stores, a measure of the number of bytes, packets, and connections that occurred between hosts during a time-period.
28. The computer system of claim 27 wherein data in the host record is organized by well known transport protocols and well-known application-level protocols.
29. The computer system of claim 28 wherein for application-level protocols and for every pair of hosts, the connection table stores statistics for traffic between hosts.
30. The computer system of claim 28 wherein the connection table stores protocol-specific records as (protocol, count) key-value pairs.