UNSUPERVISED MULTI-DIMENSIONAL COMPUTER-GENERATED LOG DATA ANOMALY DETECTION
A computer receives a stream of discrete log data entries containing at least one unique entry, for the computer to identify anomalies in machine log data. The computer generates a log data sentiment analyzer with a lexicon customized to identify a tone of content for each of said log data entries. The computer assigns a unique message ID to the unique messages and collects data attributes of the unique entries. For each unique entry, the computer identifies an entry tone or sentiment and at least one additional unique entry attribute. The computer generates a time series analysis of the identified entry sentiment and the additional attributes and conducts statistical analysis of the attributes using at least one deep learning analysis model to identify historical anomalies in the log data attributes, using the identified anomalies indicate trouble in a system associated with the collected log data.
1 . A computer-implemented method for identifying anomalies in log data, comprising:
receiving, by a computer, a plurality of discrete log data entries, said plurality having at least one unique entry;
generating, by said computer, a log data sentiment analyzer having a lexicon customized to identify a tone of content for each of said log data entries;
assigning, by said computer, a unique message ID to said at least one unique message;
collecting, by said computer, data attributes of said at least one unique entry, said attributes identifying, for each unique entry, an entry sentiment and at least one additional unique entry attribute; and
conducting, by said computer, statistical analysis of said collected data attributes using at least one deep learning analysis model to identify historical anomalies in said log data attributes,
wherein said identified anomalies indicate trouble associated with said log data.
2 . The computer-implemented method of claim 1 further including:
identifying, by said computer, an analysis time window;
conducting, by said computer, for said analysis time window, a time series analysis which identifies a negative count value associated with a quantity of log entries for which said entry sentiment is negative; and
conducting, by said computer, for said analysis time window, a time series analysis which identifies a log data characteristic selected from the group consisting of message occurrence frequency for said at least one unique message, time differences between occurrences of said unique message, and frequency of messages having a predetermined category to determine said at least one additional log data attribute.
3 . The computer-implemented method of claim 1 , wherein said at least one additional log data attribute is determined by conducting, by said computer, for said analysis time window, a time series analysis which identifies a message occurrence frequency for said at least one unique message, time differences between occurrences of said unique message, and frequency of messages having a predetermined category.
4 . The computer-implemented method of claim 1 , wherein said at least one deep learning model is selected from the group consisting of LSTM with auto-encoders, LSTM with uncertainty estimation, and LSTM with dropout.
5 . The computer-implemented method of claim 1 , wherein said at least one deep learning model is LSTM with auto-encoders, LSTM with uncertainty estimation, and LSTM with dropout.
6 . A system for identifying anomalies in log data which comprises:
a computer system comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
receive a plurality of discrete log data entries, said plurality having at least one unique entry;
generate a log data sentiment analyzer having a lexicon customized to identify a tone of content for each of said log data entries;
assign a unique message ID to said at least one unique message;
collect data attributes of said at least one unique entry, said attributes identifying, for each unique entry, an entry sentiment and at least one additional unique entry attribute; and
conduct statistical analysis of said collected data attributes using at least one deep learning analysis model to identify historical anomalies in said log data attributes, wherein said identified anomalies indicate trouble associated with said log data.
7 . The system of claim 6 further including further instructions which cause said computer to:
identify an analysis time window;
conduct for said analysis time window, a time series analysis which identifies a negative count value associated with a quantity of log entries for which said entry sentiment is negative;
and
conduct for said analysis time window, a time series analysis which identifies a log data characteristic selected from the group consisting of message occurrence frequency for said at least one unique message, time differences between occurrences of said unique message, and frequency of messages having a predetermined category to determine said at least one additional log data attribute.
8 . The system of claim 6 , wherein said at least one additional log data attribute is determined by conducting, by said computer, for said analysis time window, a time series analysis which identifies a message occurrence frequency for said at least one unique message, time differences between occurrences of said unique message, and frequency of messages having a predetermined category.
9 . The system of claim 6 , wherein said at least one deep learning model is selected from the group consisting of LSTM with auto-encoders, LSTM with uncertainty estimation, and LSTM with dropout.
10 . The system of claim 6 , wherein said at least one deep learning model is LSTM with auto-encoders, LSTM with uncertainty estimation, and LSTM with dropout.
11 . A computer program product to identify anomalies in log data, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
receive, using the computer, a plurality of discrete log data entries, said plurality having at least one unique entry;
generate, using the compouter, a log data sentiment analyzer having a lexicon customized to identify a tone of content for each of said log data entries;
assign, using the computer, a unique message ID to said at least one unique message;
collect, using the computer, data attributes of said at least one unique entry, said attributes identifying, for each unique entry, an entry sentiment and at least one additional unique entry attribute; and
conduct, using the computer, statistical analysis of said collected data attributes using at least one deep learning analysis model to identify historical anomalies in said log data attributes, wherein said identified anomalies indicate trouble associated with said log data.
12 . The computer program product of claim 11 further including further instructions which cause said computer to:
identify, using the computer, an analysis time window;
conduct, using the computer, for said analysis time window, a time series analysis which identifies a negative count value associated with a quantity of log entries for which said entry sentiment is negative; and
conduct for said analysis time window, a time series analysis which identifies a log data characteristic selected from the group consisting of message occurrence frequency for said at least one unique message, time differences between occurrences of said unique message, and frequency of messages having a predetermined category to determine said at least one additional log data attribute.
13 . The computer program product of claim 11 , wherein said at least one additional log data attribute is determined by conducting, by said computer, for said analysis time window, a time series analysis which identifies a message occurrence frequency for said at least one unique message, time differences between occurrences of said unique message, and frequency of messages having a predetermined category.
14 . The computer program product of claim 11 , wherein said at least one deep learning model is selected from the group consisting of LSTM with auto-encoders, LSTM with uncertainty estimation, and LSTM with dropout.
15 . The computer program product of claim 11 , wherein said at least one deep learning model is LSTM with auto-encoders, LSTM with uncertainty estimation, and LSTM with dropout.