System and method for detecting overlogging issues in storage systems
A method, computer program product, and computing system for processing a plurality of log files from one or more storage systems. A log pattern concerning at least two or more log files is identified by processing each log file from the plurality of log files using one or more processing threads. A potential overlogging issue associated with the log pattern is identified.
1. A computer-implemented method, executed on a computing device, comprising:
processing a plurality of log files from one or more storage systems;
identifying a log pattern concerning at least two or more log files, by processing each log file from the plurality of log files using one or more processing threads;
identifying a potential overlogging issue associated with the log pattern;
monitoring the plurality of log files being generated by the one or more storage systems for one or more log patterns; and
generating an alert indicative of an overlogging issue in response to detecting a log pattern associated with an overlogging issue.
2. The computer-implemented method of claim 1 , further comprising:
monitoring the plurality of log files generated by the one or more storage systems for one or more log patterns; and
filtering one or more log files from the plurality of log files based upon, at least in part, the one or more log patterns detected in the plurality of log files.
3. The computer-implemented method of claim 1 , further comprising:
requesting subsequent data collection from the one or more storage systems for additional log files associated with the one or more log patterns; and
filtering one or more log files from the plurality of log files based upon, at least in part, the one or more log patterns detected in the plurality of log files.
4. The computer-implemented method of claim 1 , wherein processing the plurality of log files includes dividing the log file into a plurality of log file portions.
5. The computer-implemented method of claim 1 , wherein processing each log file from the plurality of log files using the one or more processing threads includes processing each log file to identify the log pattern based upon, at least in part, one or more of:
a log file location;
a log file format;
a time window associated with the log file; and
a log line format.
6. The computer-implemented method of claim 5 , wherein processing each log file from the plurality of log files using the one or more processing threads includes aggregating results from the one or more processing threads for each log file into a plurality of queues.
7. The computer-implemented method of claim 6 , further comprising:
generating a report including the potential overlogging issue associated with the log pattern based upon, at least in part, the plurality of queues.
8. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:
processing a plurality of log files from one or more storage systems;
identifying a log pattern concerning at least two or more log files, by processing each log file from the plurality of log files using one or more processing threads;
identifying a potential overlogging issue associated with the log pattern;
monitoring the plurality of log files being generated by the one or more storage systems for one or more log patterns;
generating an alert indicative of an overlogging issue in response to detecting a log pattern associated with an overlogging issue;
requesting subsequent data collection from the one or more storage systems for additional log files associated with the one or more log patterns; and
filtering one or more log files from the plurality of log files based upon, at least in part, the one or more log patterns detected in the plurality of log files.
9. The computer program product of claim 8 , wherein the operations further comprise:
monitoring the plurality of log files generated by the one or more storage systems for one or more log patterns; and
filtering one or more log files from the plurality of log files based upon, at least in part, the one or more log patterns detected in the plurality of log files.
10. The computer program product of claim 8 , wherein processing the plurality of log files includes dividing the log file into a plurality of log file portions.
11. The computer program product of claim 8 , wherein processing each log file from the plurality of log files using the one or more processing threads includes processing each log file to identify the log pattern based upon, at least in part, one or more of:
a log file location;
a log file format;
a time window associated with the log file; and
a log line format.
12. The computer program product of claim 11 , wherein processing each log file from the plurality of log files using the one or more processing threads includes aggregating results from the one or more processing threads for each log file into a plurality of queues.
13. The computer program product of claim 12 , wherein the operations further comprise:
generating a report including the potential overlogging issue associated with the log pattern based upon, at least in part, the plurality of queues.
14. A computing system comprising:
a memory; and
a processor configured to:
process a plurality of log files from one or more storage systems,
identify a log pattern concerning at least two or more log files, by processing each log file from the plurality of log files using one or more processing threads,
identify a potential overlogging issue associated with the log pattern,
monitor the plurality of log files being generated by the one or more storage systems for one or more log patterns,
generate an alert indicative of an overlogging issue in response to detecting a log pattern associated with an overlogging issue;
request subsequent data collection from the one or more storage systems for additional log files associated with the one or more log patterns; and
filter one or more log files from the plurality of log files based upon, at least in part, the one or more log patterns detected in the plurality of log files.
15. The computing system of claim 14 , wherein the processor is further configured to:
monitor the plurality of log files generated by the one or more storage systems for one or more log patterns; and
filter one or more log files from the plurality of log files based upon, at least in part, the one or more log patterns detected in the plurality of log files.
16. The computing system of claim 14 , wherein processing the plurality of log files includes dividing the log file into a plurality of log file portions.
17. The computing system of claim 14 , wherein processing each log file from the plurality of log files using the one or more processing threads includes processing each log file to identify the log pattern based upon, at least in part, one or more of:
a log file location;
a log file format;
a time window associated with the log file; and
a log line format.
18. The computing system of claim 17 , wherein processing each log file from the plurality of log files using the one or more processing threads includes aggregating results from the one or more processing threads for each log file into a plurality of queues.