Multilevel network monitoring system architecture
View Patent ↗Network traffic information from multiple sources, at multiple time scales, and at multiple levels of detail are integrated so that users may more easily identify relevant network information. The network monitoring system stores and manipulates low-level and higher-level network traffic data separately to enable efficient data collection and storage. Packet traffic data is collected, stored, and analyzed at multiple locations. The network monitoring locations communicate summary and aggregate data to central modules, which combine this data to provide an end-to-end description of network traffic at coarser time scales. The network monitoring system enables users to zoom in on high-level, coarse time scale network performance data to one or more lower levels of network performance data at finer time scales. When high-level network performance data of interest is selected, corresponding low-level network performance data is retrieved from the appropriate distributed network monitoring locations to provide additional detailed information.
1. A system for monitoring network performance, the system comprising:
first and second network packet capture modules at first and second respective network locations, wherein each of the network packet capture modules is adapted to intercept network traffic at its network location and store first network traffic data at a first time scale;
a central network traffic data flow module at a third network location, wherein the central network traffic data flow module is adapted to receive and store first summary network traffic data from the first and second network packet capture modules, wherein the first summary network traffic data is at a second time scale coarser than the first time scale; and
wherein the central network traffic data flow module is further adapted to identify related network traffic data from the first and second network locations from the first summary network traffic data and to store references to the related network traffic data, wherein the related network traffic data is stored at the first time scale in the first and second network locations.
2. The system of claim 1 , wherein the first summary network traffic data includes first references to corresponding portions of the first network traffic data at the first time scale, wherein the central network traffic data flow module is adapted to retrieve at least one of the portions of the first network traffic data from at least one of the first and second network packet capture modules using the first references included in the first summary network traffic data.
3. The system of claim 2 , wherein the central network traffic data flow module is adapted to retrieve the at least one of the portions of the first network traffic data in response to a selection of a portion of the first summary network traffic data.
4. The system of claim 3 , wherein the selection is received via a network monitoring application.
5. The system of claim 1 , wherein the related network traffic is included in a network flow between a first source and first destination.
6. The system of claim 5 , wherein the related network traffic is included in a related network flow to at least a second destination.
7. The system of claim 1 , wherein the central network traffic data flow module is adapted to store flow traffic data derived from the first summary network traffic data.
8. The system of claim 1 , wherein the central network traffic data flow module is adapted to store macroflow traffic data derived from the first summary network traffic data.
9. The system of claim 1 , wherein the first summary network traffic data includes an attribute common to at least a portion of the first network traffic data.
10. The system of claim 1 , wherein the first summary network traffic data includes an aggregate attribute of at least a portion of the first network traffic data.
11. The system of claim 1 , wherein the first summary network traffic data includes a statistical attribute determined from at least a portion of the first network traffic data.
12. The system of claim 1 , wherein the first summary network traffic data includes a user-configured attribute of at least a portion of the first network traffic data.
13. The system of claim 12 , wherein the user-configured attribute is determined by executing user-provided code.
14. The system of claim 1 , wherein the first and second network packet capture modules store first and second portions of the first network traffic data on first and second storage devices at the first and second network locations, respectively.
15. The system of claim 14 , wherein the first storage device stores the first portion of the first network traffic data using a file system including packet data block references, wherein each packet data block reference represents a time period and includes at least one reference to a data block including a subset of the first portion of the first network traffic data intercepted within the time period.
16. The system of claim 1 , wherein the central network traffic data flow module stores first summary network traffic data on a storage device at the third network location.
17. The system of claim 1 , wherein at least one of the first and second network packet capture modules and/or the central network traffic data flow module is implemented at least partially within a virtual machine application.
18. The system of claim 1 , wherein at least one of the first and second network packet capture modules and/or the central network traffic data flow module is implemented at least partially within a network monitoring appliance.
19. The system of claim 1 , wherein the central network traffic data flow module includes an application programming interface for communicating with network management applications.
20. The system of claim 1 , wherein the central network traffic data flow module includes a scripting interface for communicating with network management applications.