Distributed network traffic data collection and storage
View Patent ↗Network traffic information from multiple sources, at multiple time scales, and at multiple levels of detail are integrated so that users may more easily identify relevant network information. The network monitoring system stores and manipulates low-level and higher-level network traffic data separately to enable efficient data collection and storage. Packet traffic data is collected, stored, and analyzed at multiple locations. The network monitoring locations communicate summary and aggregate data to central modules, which combine this data to provide an end-to-end description of network traffic at coarser time scales. The network monitoring system enables users to zoom in on high-level, coarse time scale network performance data to one or more lower levels of network performance data at finer time scales. When high-level network performance data of interest is selected, corresponding low-level network performance data is retrieved from the appropriate distributed network monitoring locations to provide additional detailed information.
1. A method of collecting and aggregating distributed traffic network data, the method comprising:
collecting first and second packet traffic data from first and second network locations, respectively;
storing first and second packet traffic data in first and second data storage devices at the first and second network locations, respectively;
generating first and second summary network traffic data at the first and second network locations, respectively, wherein the first and second summary network traffic data is at a first time scale coarser than a time scale associated with the first and second packet traffic data;
communicating first and second summary network traffic data from the first and second network locations to a third network location;
storing the first and second summary network traffic data at the third network location;
wherein the first summary network traffic data includes first references adapted to retrieve at least corresponding portions of the first packet traffic data from the first network location and the second summary network traffic data includes second references adapted to retrieve at least corresponding portions of the second packet traffic data from the second network location;
wherein the first summary network traffic data includes an attribute common to at least a portion of the first packet traffic data; and
wherein the first summary network traffic data includes an aggregate attribute of at least a portion of the first packet traffic data.
2. The method of claim 1 , wherein the first summary network traffic data includes a statistical attribute determined from at least a portion of the first packet traffic data.
3. The method of claim 1 , wherein the first summary network traffic data includes a user-configured attribute of at least a portion of the first packet traffic data.
4. The method of claim 3 , wherein the user-configured attribute is determined by executing user-provided code.
5. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method of collecting and aggregating distributed traffic network data, the method comprising:
collecting first and second packet traffic data from first and second network locations, respectively;
storing first and second packet traffic data in first and second data storage devices at the first and second network locations, respectively;
generating first and second summary network traffic data at the first and second network locations, respectively, wherein the first and second summary network traffic data is at a first time scale coarser than a time scale associated with the first and second packet traffic data;
communicating first and second summary network traffic data from the first and second network locations to a third network location;
storing the first and second summary network traffic data at the third network location;
wherein the first summary network traffic data includes first references adapted to retrieve at least corresponding portions of the first packet traffic data from the first network location and the second summary network traffic data includes second references adapted to retrieve at least corresponding portions of the second packet traffic data from the second network location; and
wherein the first summary network traffic data includes an attribute common to at least a portion of the first packet traffic data.
6. The non-transitory computer-readable storage medium of claim 5 , wherein the first summary network traffic data includes an aggregate attribute of at least a portion of the first packet traffic data.
7. The non-transitory computer-readable storage medium of claim 5 , wherein the first summary network traffic data includes a statistical attribute determined from at least a portion of the first packet traffic data.
8. The non-transitory computer-readable storage medium of claim 5 , wherein the first summary network traffic data includes a user-configured attribute of at least a portion of the first packet traffic data.
9. The non-transitory computer-readable storage medium of claim 8 , wherein the user-configured attribute is determined by executing user-provided code.
10. An apparatus, comprising:
one or more processors; and
a non-transitory computer-readable storage medium storing instructions that, when executed by the one or more processors, cause the apparatus to perform a method of collecting and aggregating distributed traffic network data, the method comprising:
collecting first and second packet traffic data from first and second network locations, respectively;
storing first and second packet traffic data in first and second data storage devices at the first and second network locations, respectively;
generating first and second summary network traffic data at the first and second network locations, respectively, wherein the first and second summary network traffic data is at a first time scale coarser than a time scale associated with the first and second packet traffic data;
communicating first and second summary network traffic data from the first and second network locations to a third network location;
storing the first and second summary network traffic data at the third network location;
wherein the first summary network traffic data includes first references adapted to retrieve at least corresponding portions of the first packet traffic data from the first network location and the second summary network traffic data includes second references adapted to retrieve at least corresponding portions of the second packet traffic data from the second network location; and
wherein the first summary network traffic data includes an attribute common to at least a portion of the first packet traffic data.
11. The apparatus of claim 10 , wherein the first summary network traffic data includes an aggregate attribute of at least a portion of the first packet traffic data.
12. The apparatus of claim 10 , wherein the first summary network traffic data includes a statistical attribute determined from at least a portion of the first packet traffic data.
13. The apparatus of claim 10 , wherein the first summary network traffic data includes a user-configured attribute of at least a portion of the first packet traffic data.
14. The apparatus of claim 13 , wherein the user-configured attribute is determined by executing user-provided code.