Enhanced identification of computer performance anomalies based on computer performance logs
In an exemplary embodiment, computer circuitry determines term characterization values for terms in computer performance logs and generates vectors that indicate the term characterization values. The computer circuitry determines vector similarity scores for these vectors. The computer circuitry aggregates the computer performance logs into aggregated logs based on the vector similarity scores. The computer circuitry selects rare logs from these aggregated logs and obtains computer performance anomaly labels for the rare logs. The computer circuitry matches new computer performance logs with the rare logs to detect the labeled computer performance anomalies.
1. A method of operating a computer to identify computer performance anomalies based on computer performance logs, the method comprising:
in the computer:
determining client term characterization values for multiple client computers;
determining client vector similarity scores for the multiple client computers;
aggregating client computer performance logs for the multiple client computers into client aggregated performance logs;
determining server term characterization values for at least one server computer;
determining server vector similarity scores for the at least one server computer;
aggregating the client aggregated performance logs into server aggregated performance logs;
determining term characterization values for terms in the computer performance logs and generating vectors indicating the term characterization values for the computer performance logs;
determining vector similarity scores for the vectors and aggregating the computer performance logs into aggregated performance logs based on the vector similarity scores and a similarity threshold;
selecting rare aggregated logs from the aggregated performance logs based on aggregation amounts for the aggregated performance logs and a rarity threshold; and
associating new computer performance logs with the rare aggregated logs to detect the computer performance anomalies.
2. The method of claim 1 wherein associating the new computer performance logs with the rare aggregated logs to detect the computer performance anomalies comprises associating a block of the new computer performance logs with a block of the rare aggregated logs to detect some of the computer performance anomalies.
3. The method of claim 1 further comprising, in the computer, removing a portion of the terms from the computer performance logs before determining the term characterization values.
4. The method of claim 1 wherein determining the term characterization values further comprises determining Term Frequency-Inverse Document Frequency (TF-IDF) values.
5. The method of claim 1 wherein determining the vector similarity scores comprises determining cosine similarity scores.
6. The method of claim 1 wherein aggregating the computer performance logs into the aggregated performance logs comprises replacing some of the terms in the aggregated performance logs with general placeholder descriptions.
7. A computer system to identify computer performance anomalies based on computer performance logs, the computer system comprising:
at least one client computer configured to:
determine client term characterization values, determine client vector similarity scores, and aggregate client computer performance logs;
determine term characterization values for terms in the computer performance logs;
generate vectors indicating the term characterization values for the computer performance logs, determine vector similarity scores for the vectors; and
aggregate the computer performance logs into aggregated performance logs based on the vector similarity scores and a similarity threshold;
at least one server computer configured to:
determine server term characterization values, determine server vector similarity scores, and aggregate the aggregated client computer performance logs into server aggregated performance logs; and
select rare aggregated logs from the aggregated performance logs based on aggregation amounts for the aggregated performance logs and a rarity threshold; and
the at least one client computer configured to associate new computer performance logs with the rare aggregated logs to detect the computer performance anomalies.
8. The computer system of claim 7 further comprising the at least one client computer configured to associate a block of the new computer performance logs with a block of the rare aggregated logs to detect some of the computer performance anomalies.
9. The computer system of claim 7 further comprising the at least one client computer configured to remove a portion of the terms from the computer performance logs before determining the term characterization values.
10. The computer system of claim 7 wherein the term characterization values comprise Term Frequency-Inverse Document Frequency (TF-IDF) values.
11. The computer system of claim 7 wherein the vector similarity scores comprise cosine similarity scores.
12. The computer system of claim 7 wherein the at least one client computer and the at least one server computer are configured to replace some of the terms in the aggregated performance logs with general placeholder descriptions.
13. A computer apparatus to identify computer performance anomalies based on computer performance logs, the computer apparatus comprising:
computer data storage configured to store log data computer processing instructions; and
the log data computer processing instructions configured to direct the computer apparatus to:
determine client term characterization values for multiple client computers, determine client vector similarity scores for the multiple client computers, and aggregate client computer performance logs for the multiple client computers into client aggregated performance logs;
determine server term characterization values for at least one server computer, determine server vector similarity scores for the at least one server computer, and aggregate the client aggregated performance logs into server aggregated performance logs;
determine term characterization values for terms in the computer performance logs;
generate vectors indicating the term characterization values for the computer performance logs;
determine vector similarity scores for the vectors;
aggregate the computer performance logs into aggregated performance logs based on the vector similarity scores and a similarity threshold;
select rare aggregated logs from the aggregated performance logs based on aggregation amounts for the aggregated performance logs and a rarity threshold; and
associate new computer performance logs with the rare aggregated logs to detect the computer performance anomalies.
14. The computer apparatus of claim 13 wherein the log data computer processing instructions are configured to direct the computer apparatus to associate a block of the new computer performance logs with a block of the rare aggregated logs to detect some of the computer performance anomalies.
15. The computer apparatus of claim 13 wherein the log data computer processing instructions are configured to direct the computer apparatus to remove a portion of the terms from the computer performance logs before determining the term characterization values.
16. The computer apparatus of claim 13 wherein the term characterization values comprise Term Frequency-Inverse Document Frequency (TF-IDF) values.
17. The computer apparatus of claim 13 wherein the vector similarity scores comprise cosine similarity scores.
18. The computer apparatus of claim 13 wherein the log data computer processing instructions are configured to direct the computer apparatus to replace some of the terms in the aggregated performance logs with general placeholder descriptions.