Method and apparatus for retrieving and combining summarized log data in a distributed log data processing system
A system and method is disclosed for collecting, storing and reporting raw log data from log-producing devices such as firewalls and routers. The log-producing devices may be both local and remote—i.e., linked to a raw log server via a LAN and/or a WAN. A log data analyzer at a remote location gathers log data from devices at that remote location into time-defined sets and then sends those sets over a WAN (which may be the Internet) to a raw log server using a first protocol. Local log-producing devices may send their log data to the log data analyzer via a LAN using a second protocol. The log data analyzer forwards the raw log data local devices to an appropriate log data analyzer for parsing, summarizing and storage in one or more databases. The raw log server combines local and remote sets of raw log data for a given time period and stores them in a storage area of raw log data. A central management station is used to query the various databases in the system and to merge database reports into a single report for display.
1 . A method for processing log data from a plurality of log-producing devices which is stored as sunmarized log data in a plurality of log data analyzers, the method comprising:
formulating database queries in a management station;
sending database queries from the management station to a plurality of log data analyzers;
receiving in the management station a plurality of reports from the plurality of log data analyzers; and
combining in the management station the plurality of reports into a single report.
2 . A method for processing log data from a plurality of log-producing devices which is stored as summarized log data in a plurality of log data analyzers, the method comprising:
associating a database query with a certain log data analyzer;
sending a database query from a management station to the certain log data analyzer;
receiving in the management station a report from a database maintained by the certain log data analyzer; and,
displaying the report on the management station.
3 . A method for processing log data from a plurality of log-producing devices which is stored as summarized log data in a plurality of log data analyzers, the method comprising:
associating a database query with certain of the plurality of log data analyzers;
sending database queries from a management station to each of the certain log data analyzers;
receiving in the management station a plurality of reports from databases maintained by the certain log data analyzers;
merging the plurality of reports into a single report; and,
displaying the merged report on the management station.
4 . A method as recited in claim 3 wherein associating a database query with a certain log data analyzer is performed by table look up on a raw log server.
5 . A data processing system for processing log data from a plurality of log-producing devices which comprises:
a plurality of log data analyzers storing summarized log data; and,
a management station connected to the plurality of log data analyzers via data communications links and which
formulates database queries;
sends database queries to a plurality of log data analyzers;
receives a plurality of reports from the plurality of log data analyzers; and
combines the plurality of reports into a single report.
6 . A data processing system for processing log data from a plurality of log-producing devices which comprises:
a plurality of log data analyzers storing summarized log data; and,
a management station connected to the plurality of log data analyzers via data communications links and which
associates a database query with a certain log data analyzer;
sends a database query to the certain log data analyzer;
receives a report from a database maintained by the certain log data analyzer; and,
displays the report.
7 . A data processing system for processing log data from a plurality of log-producing devices which comprises:
a plurality of log data analyzers storing summarized log data; and,
a management station in data communication with the plurality of log data analyzers and which
associates a database query with certain of the plurality of log data analyzers;
sends database queries to each of the certain log data analyzers;
receives a plurality of reports from databases maintained by the certain log data analyzers;
merges the plurality of reports into a single report; and,
displays the merged report.
8 . A data processing system as recited in claim 7 wherein the management station associates a database query with certain of the plurality of log data analyzers by table look up on a raw log server.