Computer log retrieval based on multivariate log time series

1. A method for computer log retrieval, comprising:

parsing, by a processor device, each log of a set of logs, and comparing each log against a model database, wherein parsing includes applying a clustering process and pattern recognition to unparsed logs from the set of logs to produce new log formats;

frequency counting each of one or more log formats of the set of logs to obtain single time series;

forming a multivariate time series from the single time series according to log message frequencies within a selected time resolution for each log format in the set of logs;

obtaining a time series signature for the set of logs based on the multivariate time series; and

storing and indexing the time series signature in the model database in a query accessible format.

2. The method as recited in claim 1 , wherein the new log formats include log formats that do not exist in an aggregation of all previous log formats.

3. The method as recited in claim 1 , wherein frequency counting for each log format of the set of logs to obtain single time series further comprises:

counting a number of log messages in each of a plurality of time ranges for each log format; and

generating the single time series based on the number of log messages in each of the plurality of time ranges.

4. The method as recited in claim 1 , wherein frequency counting further comprises:

setting a starting point for frequency counting based on a common reference in both a learning stage and a query stage; and

forming consecutive time periods based on a time range parameter and the common reference.

5. The method as recited in claim 1 , further comprising:

outputting a final multiple sets of multivariate log format time series for the set of logs to represent the set of logs.

6. The method as recited in claim 1 , wherein the multi-variate time series includes log system dynamics of the set of logs.

7. The method as recited in claim 1 , wherein each of at least one item and each of at least one entry in the database is a key-value pair and wherein a key is an index of the set of logs and a value is a corresponding joint syntactic and semantic feature vector.

8. The method as recited in claim 1 , further comprising:

retrieving computer logs based on log system dynamics information of the set of logs.

9. A method for computer log retrieval, comprising:

receiving a set of query logs;

transforming, by a processor device, the set of query logs into a query log multi-variate time series;

accessing log multivariate time series of historical logs;

computing and ranking a similarity distance between the query log multivariate time series and the log multivariate time series of the historical logs;

providing a final similarity score including a summation of all similarity distances divided by a total number of distinct log formats between sets of multivariate time series; and

retrieving a highest ranked set of historical logs, the highest ranked set being a most similar set of logs compared to the set of query logs.

10. The method as recited in claim 9 , further comprising:

presenting the retrieved set of logs with extracted statistics.

11. The method as recited in claim 9 , wherein accessing the log multivariate time series of the historical logs further comprises:

determining the log multivariate time series of the historical logs.

12. The method as recited in claim 9 , further comprising:

ranking the similarity distance based on a cosine similarity.

13. The method as recited in claim 9 , further comprising:

extracting an entire log signature multivariate log format time series database with log format identifiers and corresponding time series; and

writing the entire log signature multivariate log format time series database into a file for subsequent processing.

14. The method as recited in claim 9 , further comprising:

identifying variables A and B associated with the set of query logs and the historical logs;

determining a cosine similarity, cos(θ), using a dot product and magnitude represented as

cos

⁡

(

θ

)

=

A

·

B



A



⁢

⁢



B



=

∑

i

=

1

n

⁢

A

i

⁢

B

i

∑

i

=

1

n

⁢

A

i

2

⁢

∑

i

=

1

n

⁢

B

i

2

,

wherein each A i and B i are term frequencies of log message counts within a particular time period.

15. The method as recited in claim 9 , wherein transforming the set of query logs into the query log multi-variate time series further comprises:

generating the query log multi-variate time series by sliding a time partition window for each of a plurality of time periods to a limit of a time range parameter.

16. The method as recited in claim 9 , further comprising:

determining a maximum value of self-similarity of a normal historical log of the set of historical logs; and

detecting an anomaly in response to determining that a value of similarity of the query logs to the set of historical logs is greater than the maximum value of self-similarity.

17. The method as recited in claim 9 , wherein the similarity distance is defined as a log formats multivariate time series similarity between sets of logs.

18. A computer system for computer log retrieval, comprising:

a processor device operatively coupled to a memory device, the processor device being configured to:

receive a set of query logs;

transform the set of query logs into a query log multi-variate time series;

access a log multivariate time series of historical logs;

compute and rank a similarity distance between the query log multivariate time series and the log multivariate time series of the historical logs;

provide a final similarity score including a summation of all similarity distances divided by a total number of distinct log formats between sets of multivariate time series; and

retrieve a highest ranked set of historical logs as a most similar set of logs compared to the set of query logs.