Machine learned decision guidance for alerts originating from monitoring systems
Described herein is a system and method for determining whether a detected issue in a computing system is a bug introduced by a developer or an intermittent issue. When an issue is detected, information about the issue is received. A determination is then made as to whether the issue is a new issue or whether it has been previously detected. When it is determined that the issue is a new issue, information about the issue is stored in a storage device. A feature vector is generated for the issue and is analyzed to determine a probability that the issue is a bug. When it is determined that the issue is a bug, the occurrence of the issue is reported to a user of the system that can correct the bug. Once the bug is corrected, the correction is provided back to the system and is used to train the system.
1. A method for automatically determining whether a detected issue in a computing system is a bug, comprising:
receiving information about a detected issue;
storing the information about the detected issue in a storage device;
generating a feature vector for the detected issue;
analyzing the feature vector to determine a probability that the detected issue is a bug caused by an alteration of at least a portion of source code used by the computing system, wherein the analysis is based, at least in part, on the feature vector;
when the probability is above a threshold, reporting the detected issue to a client device;
receiving input corresponding to the detected issue, the input indicating whether the detected issue is a bug;
using the input to update the information about the detected issue in the storage device; and
providing the updated information to a system that identified the detected issue to automatically train the system.
2. The method of claim 1 , wherein the input comprises a solution to the detected issue.
3. The method of claim 2 , further comprising updating the analysis based, at least in part, on the received solution to the detected issue.
4. The method of claim 2 , further comprising:
associating the solution with the detected issue; and
storing the solution in the storage device.
5. The method of claim 1 , further comprising prioritizing the detected issue with respect to one or more additional detected issues.
6. The method of claim 1 , wherein generating a feature vector comprises performing one or more transformations on the information about the detected issue.
7. The method of claim 6 , wherein the one or more transformations are selected from a group comprising: normalization, counting occurrences of key words from a stack trace, and binning.
8. A system, comprising:
at least one processing unit; and
a memory storing computer executable instructions that, when executed by the at least one processing unit, cause the system to perform a method for detecting occurrences of issues in a software system, comprising:
receiving information about a detected issue;
comparing a fingerprint of the detected issue to one or more fingerprints of previously detected issues to determine whether the detected issue is new;
when it is determined that the detected issue is new:
storing the information about the detected issue in a storage device;
generating a feature vector for the detected issue;
analyzing the feature vector to determine a probability that the detected issue is a bug caused by an alteration of at least a portion of source code used by the system;
providing an ordered list of detected issues to a client device;
receiving input corresponding one or more of the detected issues in the ordered list of detected issues, the input indicating whether the detected issue is a bug or a false positive; and
using the input to update the information about the detected issue.
9. The system of claim 8 , further comprising instructions for using one or more models to evaluate a received feature vector based, at least in part, on information contained in the feature vector.
10. The system of claim 8 , further comprising instructions for receiving a solution to the detected issue.
11. The system of claim 10 , further comprising instructions for associating the solution with the detected issue.
12. The system of claim 10 , further comprising instructions for updating one or more models based, at least in part, on the solution.
13. The system of claim 8 , wherein an order of the ordered list is based on the determined probability.
14. The system of claim 8 , further comprising instructions for assigning a priority to the detected issue in view of previously detected issues.
15. The system of claim 8 , further comprising instructions for storing information about the detected issue in the storage device when it is determined that the detected issue is not a bug.
16. A computer-readable storage medium storing computer executable instructions that, when executed by a processing unit, causes the processing unit to perform a method, comprising:
generating a feature vector for a detected issue using information associated with the detected issue;
analyzing the feature vector using one or more models to determine a probability that the detected issue is a bug caused by an alteration of at least a portion of source code;
reporting the occurrence of the detected issue;
receiving input that indicates whether the detected issue is a false positive or a bug;
receiving a solution to the detected issue when the input indicates the detected issue is a bug; and
updating at least one of the one or more models using the solution to the detected issue.
17. The computer-readable storage medium of claim 16 , further comprising instructions for storing the solution in a storage device.
18. The computer-readable storage medium of claim 16 , further comprising instructions for associating the solution with the detected issue.
19. The computer-readable storage medium of claim 16 , further comprising instructions for storing the information about the detected issue.
20. The computer-readable storage medium of claim 16 , further comprising instructions for determining which model of the one or more models is configured to analyze the feature vector.