Extended dynamic intelligent log analysis tool
This disclosure describes implementations for categorizing and displaying a Gantt chart of categorized errors, as well as performing an odd pattern analysis to identify anomalous errors including obtaining a log file that includes a number of log entries where each log entry includes an error message. Converting each error message into an error vector using a predictive model. Determining a category for each error vector using a K-nearest neighbor algorithm, and generating a Gantt chart representing error messages arranged hierarchically by category as a function of time for display at a user interface.
1. A computer implemented method comprising:
obtaining, from a network during a period of minimal network activity, and by a computer system coupled to the network through a network interface device, a log file comprising a plurality of log entries, each log entry comprising an error message;
converting, by the computer system and using a predictive model, each error message into an error vector;
determining, by the computer system and using a K-nearest neighbors algorithm, a category for each error vector; and
generating, by the computer system and for display in a user interface, a Gantt chart representing error messages arranged hierarchically by category as a function of time.
2. The method of claim 1 , wherein the predictive model is an N-gram model or a Sent2Vec algorithm.
3. The method of claim 2 , wherein the predictive model is selected based on performance criteria.
4. The method of claim 3 , wherein the performance criteria comprise a determination of topic coherence.
5. The method of claim 1 , comprising:
parsing the log file to extract each error message from the plurality of log entries; and
converting each error message into an error vector by generating, for each error message, an error object to be converted into the error vector, the error object comprising at least one attribute associated with an error.
6. The method of claim 5 , wherein at least one attribute comprises a date and time of the error, and a text phrase associated with the error.
7. The method of claim 1 , comprising transmitting the error messages and determined categories to a backend system for training of one or more machine learning models.
8. The method of claim 7 , wherein the error messages and categories are transmitted during a period of low network activity, and wherein training the one or more machine learning models comprises an N-gram model, a Sent2Vec algorithm, or a K-nearest neighbor algorithm.
9. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:
obtaining, from a network through a network interface device during a period of minimal network activity, a log file comprising a plurality of log entries, each log entry comprising an error message;
converting, using a predictive model, each error message into an error vector;
determining, using a K-nearest neighbors algorithm, a category for each error vector; and
generating, for display in a user interface, a Gantt chart representing error messages arranged hierarchically by category as a function of time.
10. The computer-readable medium of claim 9 , wherein the predictive model is an N-gram model or a Sent2Vec algorithm.
11. The computer-readable medium of claim 10 , wherein the predictive model is selected based on performance criteria.
12. The computer-readable medium of claim 11 , wherein the performance criteria comprise a determination of topic coherence.
13. The computer-readable medium of claim 9 , comprising:
parsing the log file to extract each error message from the plurality of log entries; and
converting each error message into an error vector by generating, for each error message, an error object to be converted into the error vector, the error object comprising at least one attribute associated with an error.
14. The computer-readable medium of claim 13 , wherein at least one attribute comprises a date and time of the error, and a text phrase associated with the error.
15. The computer-readable medium of claim 9 , comprising transmitting the error messages and determined categories to a backend system for training of one or more machine learning models.
16. The computer-readable medium of claim 15 , wherein the error messages and categories are transmitted during a period of low network activity, and wherein training the one or more machine learning models comprises an N-gram model, a Sent2Vec algorithm, or a K-nearest neighbor algorithm.
17. A computer-implemented system, comprising:
one or more computers; and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising:
obtaining, from a network through a network interface device during a period of minimal network activity, a log file comprising a plurality of log entries, each log entry comprising an error message;
converting, using a predictive model, each error message into an error vector;
determining, using a K-nearest neighbors algorithm, a category for each error vector; and
generating, for display in a user interface, a Gantt chart representing error messages arranged hierarchically by category as a function of time.
18. The system of claim 17 , wherein the predictive model is an N-gram model or a Sent2Vec algorithm.
19. The system of claim 18 , wherein the predictive model is selected based on performance criteria.
20. The system of claim 19 , wherein the performance criteria comprise a determination of topic coherence.