METHOD AND APPARATUS FOR OPTIMIZING LOG FILE FILTERING
A method and apparatus for optimizing list filtering comprising receiving a list of items from one or more servers, matching the list of items against a set of filters, ordering the set of filters based on the frequency of matches for the set of filters for each filter in the set of filters and applying the ordered set of filters for matching on a next received list of items.
1 . A method for optimizing list filtering comprising:
receiving a list of items from one or more servers;
matching the list of items against a set of filters;
ordering the set of filters based on the frequency of matches for the set of filters for each filter in the set of filters; and
applying the ordered set of filters for matching on a next received list of items.
2 . The method of claim 1 , further comprising:
reordering each filter in the set of filters after a predetermined number of items have been received and matched.
3 . The method of claim 1 , further comprising:
reordering each filter in the set of filters at scheduled times of day.
4 . The method of claim 1 further comprising:
receiving one or more entities which match frequently occurring items to be ignored; and
adding an ignore filter to the set of filters for matching the frequently occurring items to reduce processing time by ordering the ignore filter before other filters in the set of filters.
5 . The method of claim 1 further comprising:
prior to matching the list of items against a set of filters, removing all lines matching one or more entities which match frequently occurring items to be ignored.
6 . The method of claim 1 further comprising:
generating one or more new filters based on entities which match frequency occurring items to be ignored; and
adding the one or more new filters to the set of filters.
7 . The method of claim 1 wherein each item in the list of items is a line from a textual log file and each line represents an event in a VoIP network.
8 . The method of claim 6 wherein each line matches at most one filter.
9 . The method of claim 6 further comprising:
receiving the list of items streamed from the tail end of the textual log file.
10 . The method of claim 6 further comprising:
receiving the textual log file from the server and then performing the matching on the entirety of the textual log file.
11 . An ordering apparatus for optimizing list filtering comprising:
a filtering module for receiving a list of items from one or more server, the filter module further comprising a filter processor for matching the list of items against a set of filters; and
a sorting module, coupled to the filter module, for ordering the set of filters based on the frequency of matches for each filter in the set of filters and applying the ordered set of filters for matching on a next received list of items.
12 . The apparatus of claim 11 , wherein the filtering module further reorders the set of filters after a predetermined number of items have been received and matched.
13 . The apparatus of claim 11 , wherein the filtering module further reorders each filter in the set of filters at scheduled times of day.
14 . The apparatus of claim 11 wherein the filtering module further:
receives one or more entities which match frequently occurring items to be ignored; and
adds an ignore filter to the set of filters for matching the frequently occurring items to reduce processing time by ordering the ignore filter before other filters in the set of filters.
15 . The apparatus of claim 11 wherein the filtering module is configured further for: prior to matching the list of items against a set of filters, removing all lines matching one or more entities which match frequently occurring items to be ignored.
16 . The apparatus of claim 11 , wherein the filtering module is configured further for generating one or more new filters based on entities which match frequency occurring items to be ignored and adding the one or more new filters to the set of filters.
17 . The apparatus of claim 11 wherein each item in the list of items is a line from a textual log file and each line represents an event in a VoIP network.
18 . The apparatus of claim 16 wherein each line matches at most one filter.
19 . The apparatus of claim 16 wherein the filtering module is further configured for receiving the list of items streamed from the tail end of the textual log file.
20 . The apparatus of claim 16 wherein the filtering module is further configured for:
receiving the textual log file from the server and then performing the matching on the entirety of the textual log file.