Anomaly recognition in information technology environments
A method comprises obtaining a set of log files for a software system. The set of log files applies to an extended window. A periodic pattern in a first set of error-event surges in the set of log files is identified. The error-event surges in the first set is identified as event noise. A second set of log files for the software system is obtained. The second set of log files applies to a shortened window. Timeseries analysis on the second set of log files is performed. A particular error-event surge in a detection period in the second set of log files that is abnormal as compared to the shortened window is detected based on the timeseries analysis. That the particular error-event surge does not fit into the periodic pattern is determined, the particular error-event surge is characterized as an anomaly, based on the determining.
1. A method comprising:
obtaining a first set of log files for a software system, wherein the first set of log files applies to a shortened window;
detecting, a particular error-event surge in a detection period in the first set of log files that is abnormal as compared to the shortened window;
determining that the particular error-event surge does not fit a periodic pattern of error-event surges in an extended window; and
characterizing, based on the determining, the particular error-event surge as an anomaly.
2. The method of claim 1 , wherein the characterizing comprises defining an anomaly window for the anomaly.
3. The method of claim 2 , wherein defining the anomaly window comprises merging an activity period in which the particular error-event surge occurred with an adjacent period.
4. The method of claim 1 , further comprising collecting event logs for the anomaly.
5. The method of claim 1 , further comprising aggregating and normalizing the first set of log files.
6. A system comprising:
a processor; and
a memory in communication with the processor, the memory containing program instructions that, when executed by the processor, are configured to cause the processor to perform a method, the method comprising:
obtaining a first set of log files for the software system, wherein the first set of log files applies to a shortened window;
detecting a particular error-event surge in a detection period in the first set of log files that is abnormal as compared to the shortened window;
determining that the particular error-event surge does not fit a periodic pattern of error-event surges in an extended window; and
characterizing, based on the determining, the particular error-event surge as an anomaly.
7. The system of claim 6 , wherein the characterizing comprises defining an anomaly window for the anomaly.
8. The system of claim 7 , wherein defining the anomaly window comprises merging an activity period in which the particular error-event surge occurred with an adjacent period.
9. The system of claim 6 , wherein the method performed by the system further comprises collecting event logs for the anomaly.
10. The system of claim 6 , wherein the method performed by the system further comprises aggregating and normalizing the first set of log files.
11. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
obtain a first set of log files for a software system, wherein the first set of log files applies to a shortened window;
detect a particular error-event surge in a detection period in the first set of log files that is abnormal as compared to the shortened window;
determine that the particular error-event surge does not fit a periodic pattern of error-event surges in an extended window; and
characterize, based on the determining, the particular error-event surge as an anomaly.
12. The computer program product of claim 11 , wherein the characterizing comprises defining an anomaly window for the anomaly.
13. The computer program product of claim 11 , wherein defining the anomaly window comprises merging an activity period in which the particular error-event surge occurred with an adjacent period.
14. The computer program product of claim 11 , wherein the program instructions further cause the computer to aggregate and normalize the first set of log files.
15. The method of claim 1 , further comprising:
obtaining a second set of log files for a software system, wherein the second set of log files applies to the extended window; and
identifying the periodic pattern in a set of error-event surges in the second set of log files.
16. The system of claim 6 , wherein the method performed by the system further comprises:
obtaining a second set of log files for a software system, wherein the second set of log files applies to the extended window; and
identifying the periodic pattern in a set of error-event surges in the second set of log files.
17. The system of claim 11 , wherein the program instructions further cause the computer to:
obtain a second set of log files for a software system, wherein the second set of log files applies to the extended window; and
identify the periodic pattern in a set of error-event surges in the second set of log files.
18. The computer program product of claim 17 , wherein the identifying comprises inserting the second set of log files into a sequence detection algorithm.
19. The method of claim 15 , wherein the identifying comprises inserting the second set of log files into a sequence detection algorithm.
20. The system of claim 16 , wherein the identifying comprises inserting the second set of log files into a sequence detection algorithm.