Method and apparatus for processing test execution logs to detremine error locations and error types
A method of processing test execution logs to determine error location and source includes creating a set of training examples based on previously processed test execution logs, clustering the training examples into a set of clusters using an unsupervised learning process, and using training examples of each cluster to train a respective supervised learning process to label data where each generated cluster is used as a class/label to identify the type of errors in the test execution log. The labeled data is then processed by supervised learning processes, specifically a classification algorithm. Once the classification model is built it is used to predict the type of the errors in future/unseen test execution logs. In some embodiments, the unsupervised learning process is a density-based spatial clustering of applications with noise clustering application, and the supervised learning processes are random forest deep neural networks.
1. A method of processing test execution logs to determine error locations and types, comprising:
creating training examples from labeled test execution logs, the test execution logs describing actions that occurred in connection with execution of one or more tests on a storage system;
clustering the training examples into two or more clusters based on features of the test execution logs using an unsupervised learning process;
using the training examples of each cluster to train a separate respective supervised learning process for the cluster to learn a regression between features of the test execution logs and error types associated with the labels;
assigning a new unlabeled test execution log to one of the clusters based on features of the new unlabeled test execution log; and
using the trained supervised learning process for the respective one of the clusters to determine a label for the new unlabeled test execution log.
2. The method of claim 1 , wherein the step of clustering is implemented using a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering process.
3. The method of claim 1 , wherein creating training examples comprises parsing the test execution logs, performing a keyword identification on each of the test execution logs, constructing a feature space based on the keyword identification, and performing a similarity comparison between the training examples.
4. The method of claim 3 , wherein the step of performing the keyword identification comprises extracting the features from the test execution logs using a statistical text mining algorithm.
5. The method of claim 4 , wherein the statistical text mining algorithm is a bag-of-words algorithm.
6. The method of claim 1 , wherein each separate respective supervised learning process is a deep neural network.
7. The method of claim 6 , wherein each separate respective supervised learning process is a random forest of deep neural networks.
8. The method of claim 1 , wherein the label is an error type and location of an error describe by the new unlabeled test execution log.
9. The method of claim 1 , further comprising attaching the determined label to the new unlabeled test execution log.
10. A non-transitory tangible computer readable storage medium having stored thereon a computer program for processing test execution logs to determine error locations and types, the computer program including a set of instructions which, when executed by a computer, cause the computer to perform a method comprising the steps of:
creating training examples from labeled test execution logs, the test execution logs describing actions that occurred in connection with execution of one or more tests on a storage system;
clustering the training examples into two or more clusters based on features of the test execution logs using an unsupervised learning process;
using the training examples of each cluster to train a separate respective supervised learning process for the cluster to learn a regression between features of the test execution logs and error types associated with the labels;
assigning a new unlabeled test execution log to one of the clusters based on features of the new unlabeled test execution log; and
using the trained supervised learning process for the respective one of the clusters to determine a label for the new unlabeled test execution log.
11. The non-transitory tangible computer readable storage medium of claim 10 , wherein the step of clustering is implemented using a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering process.
12. The non-transitory tangible computer readable storage medium of claim 10 , wherein creating training examples comprises parsing the test execution logs, performing a keyword identification on each of the test execution logs, constructing a feature space based on the keyword identification, and performing a similarity comparison between the training examples.
13. The non-transitory tangible computer readable storage medium of claim 12 , wherein the step of performing the keyword identification comprises extracting the features from the test execution logs using a statistical text mining algorithm.
14. The non-transitory tangible computer readable storage medium of claim 13 , wherein the statistical text mining algorithm is a bag-of-words algorithm.
15. The non-transitory tangible computer readable storage medium of claim 10 , wherein each separate respective supervised learning process is a deep neural network.
16. The non-transitory tangible computer readable storage medium of claim 15 , wherein each separate respective supervised learning process is a random forest of deep neural networks.
17. The non-transitory tangible computer readable storage medium of claim 10 , wherein the label is an error type and location of an error describe by the new unlabeled test execution log.
18. The non-transitory tangible computer readable storage medium of claim 10 , further comprising attaching the determined label to the new unlabeled test execution log.