Log-based computer failure diagnosis
Methods and systems for system failure diagnosis and correction include extracting syntactic patterns from a plurality of logs with heterogeneous formats. The syntactic patterns are clustered according to categories of system failure. A single semantically unique pattern is extracted for each category of system failure. The semantically unique patterns are matched to recent log information to detect a corresponding system failure. A corrective action us performed responsive to the detected system failure.
1. A method for system failure diagnosis and correction, comprising:
extracting syntactic patterns from a plurality of logs with heterogeneous formats;
clustering the syntactic patterns according to categories of system failure;
extracting a single semantically unique pattern for each category of system failure;
matching the semantically unique patterns to recent log information using a processor to detect a corresponding system failure; and
performing a corrective action responsive to the detected system failure.
2. The method of claim 1 , wherein the syntactic patterns represent structural information regarding the log formats.
3. The method of claim 1 , wherein the semantic patterns represent log content information regarding the respective categories of system failure.
4. The method of claim 1 , wherein extracting the syntactic patterns comprises embedding each log in a latent vector space.
5. The method of claim 4 , wherein a distance metric in the latent vector space represents a degree of similarity between respective syntactic patterns.
6. The method of claim 1 , further comprising generating system failure signatures from the semantically unique patterns using automata models that represent sequences of log patterns.
7. The method of claim 6 , wherein matching the semantically unique patterns to recent log information comprises identifying sequences in the recent log information that match one or more failure signatures.
8. The method of claim 6 , wherein the failure signatures comprise time-order information in the sequences of log patterns.
9. The method of claim 1 , further comprising diagnosing the detected system failure by matching the semantically unique patterns to recent log information.
10. The method of claim 1 , wherein the corrective action includes at least one of the group consisting of shutting down a system, activating a backup system, rebooting a malfunctioning system, changing power supply settings, changing cooling settings, and changing security settings.
11. A failure diagnosis and correction system, comprising:
a failure signature module configured to extract syntactic patterns from a plurality of logs with heterogeneous formats, to cluster the syntactic patterns according to categories of system failure, and to extract a single semantically unique pattern for each category of system failure;
a failure diagnosis module comprising a processor configured to match the semantically unique patterns to recent log information to detect a corresponding system failure; and
a corrective module configured to perform a corrective action responsive to the detected system failure.
12. The system of claim 11 , wherein the syntactic patterns represent structural information regarding the log formats.
13. The system of claim 11 , wherein the semantic patterns represent log content information regarding the respective categories of system failure.
14. The system of claim 11 , wherein the failure signature module is further configured to embed each log in a latent vector space.
15. The system of claim 14 , wherein a distance metric in the latent vector space represents a degree of similarity between respective syntactic patterns.
16. The system of claim 11 , wherein the failure signature module is further configured to generate system failure signatures from the semantically unique patterns using automata models that represent sequences of log patterns.
17. The system of claim 16 , wherein the failure diagnosis module is further configured to identify sequences in the recent log information that match one or more failure signatures.
18. The system of claim 16 , wherein the failure signatures comprise time-order information in the sequences of log patterns.
19. The system of claim 11 , wherein the failure diagnosis module is further configured to diagnose the detected system failure by matching the semantically unique patterns to recent log information.
20. The system of claim 11 , wherein the corrective action includes at least one of the group consisting of shutting down a system, activating a backup system, rebooting a malfunctioning system, changing power supply settings, changing cooling settings, and changing security settings.