Systems and methods for labeling automatically generated reports
The disclosed computer-implemented method for labeling automatically generated reports may include (i) identifying incident reports that describe incidents that each involve at least one computing system and that comprise automatically collected information about the incidents and a manually analyzed subset of incident reports that comprise manually generated information, (ii) assigning at least one label to at least one incident report in the manually analyzed subset based on applying a machine learning model to the manually generated information, (iii) deriving, from the automatically collected information, a set of features that describe incident reports, (iv) propagating at least one label from a labeled incident report to an incident report that is not in the manually analyzed subset and that comprises similar features with the labeled incident report, and (v) performing an action related to the label on the incident report. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for labeling automatically generated reports, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
identifying:
a set of incident reports that describe incidents that each involve at least one computing system and that comprise automatically collected information about the incidents; and
a manually analyzed subset of the set of incident reports that further comprise manually generated information about the incidents in addition to automatically collected information about the incidents;
assigning at least one label from a set of labels to at least one incident report in the manually analyzed subset of the set of incident reports, wherein the set of labels was generated based on applying a machine learning model to the manually generated information;
deriving, from the automatically collected information, a set of features that describe the set of incident reports wherein each feature in the set of features discriminates between differing types of incident reports;
plotting each incident report in the set of incident reports on a graph based on at least one value for the incident report of at least one feature in the set of features;
in response to determining that the labeled incident report and the incident report comprise similar features from the set of features by determining that the labeled incident report and the incident report are plotted within a predetermined distance of each other in the graph, propagating at least one label assigned to a labeled incident report in the manually analyzed subset of the set of incident reports to an incident report that is not in the manually analyzed subset; and
performing an action related to the label on the incident report that is not in the manually analyzed subset in response to propagating the label to the incident report.
2. The computer-implemented method of claim 1 , wherein performing the action comprises forwarding the incident report to an analyst for manual analysis.
3. The computer-implemented method of claim 1 , wherein performing the action comprises propagating at least one manually added note from the labeled incident report to the incident report.
4. The computer-implemented method of claim 1 , wherein the incidents comprise security incidents and the set of incident reports comprises incident reports generated by security applications.
5. The computer-implemented method of claim 1 , wherein assigning the at least one label from a set of labels to the at least one incident report in the manually analyzed subset of the set of incident reports based on applying the machine learning model to the manually generated information comprises analyzing the manually generated information using a natural language processing technique.
6. The computer-implemented method of claim 1 , wherein deriving, from the automatically collected information, the set of features that describe the set of incident reports comprises:
deriving an original set of features from the set of incident reports; and
refining, using a machine learning model, the original set of features into a reduced set of features that comprises a subset of the original set features that differentiates between incident reports of different types more effectively than a subset of the original features not in the reduced set of features.
7. The computer-implemented method of claim 1 , wherein assigning the at least one label from a set of labels to the at least one incident report in the manually analyzed subset of the set of incident reports comprises:
labeling the at least one incident report in the manually analyzed subset as severe; and
labeling at least one additional incident report in the manually analyzed subset as not severe.
8. The computer-implemented method of claim 7 , wherein performing the action comprises at least one of:
forwarding the incident report labeled as severe to an analyst for manual analysis in response to labeling the incident report as severe; and
avoiding forwarding the additional incident report labeled as not severe to an analyst for manual analysis in response to labeling the additional incident report as not severe.
9. A system for labeling automatically generated reports, the system comprising:
an identification module, stored in memory, that identifies:
a set of incident reports that describe incidents that each involve at least one computing system and that comprise automatically collected information about the incidents; and
a manually analyzed subset of the set of incident reports that further comprise manually generated information about the incidents in addition to automatically collected information about the incidents;
an assignment module, stored in memory, that assigns at least one label from a set of labels to at least one incident report in the manually analyzed subset of the set of incident reports, wherein the set of labels was generated based on applying a machine learning model to the manually generated information;
a deriving module, stored in memory, that derives, from the automatically collected information, a set of features that describe the set of incident reports, wherein each feature in the set of features discriminates between differing types of incident reports;
a propagation module, stored in memory, that propagates at least one label assigned to a labeled incident report in the manually analyzed subset of the set of incident reports to an incident report that is not in the manually analyzed subset and that comprises similar features from the set of features with the labeled incident report by:
plotting each incident report in the set of incident reports on a graph based on at least one value for the incident report of at least one feature in the set of features; and
propagating the at least one label from the labeled incident report to the incident report in response to determining that the labeled incident report and the incident report are plotted within a predetermined distance of each other in the graph;
a performing module, stored in memory, that performs an action related to the label on the incident report that is not in the manually analyzed subset in response to propagating the label to the incident report; and
at least one physical processor configured to execute the identification module, the assignment module, the deriving module, the propagation module, and the performing module.
10. The system of claim 9 , wherein the performing module performs the action by forwarding the incident report to an analyst for manual analysis.
11. The system of claim 9 , wherein the performing module performs the action by propagating at least one manually added note from the labeled incident report to the incident report.
12. The system of claim 9 , wherein the incidents comprise security incidents and the set of incident reports comprises incident reports generated by security applications.
13. The system of claim 9 , wherein the assignment module assigns the at least one label from a set of labels to the at least one incident report in the manually analyzed subset of the set of incident reports based on applying the machine learning model to the manually generated information by analyzing the manually generated information using a natural language processing technique.
14. The system of claim 9 , wherein the deriving module derives, from the automatically collected information, the set of features that describe the set of incident reports by:
deriving an original set of features from the set of incident reports; and
refining, using a machine learning model, the original set of features into a reduced set of features that comprises a subset of the original set features that differentiates between incident reports of different types more effectively than a subset of the original features not in the reduced set of features.
15. The system of claim 9 , wherein the assignment module assigns the at least one label from a set of labels to the at least one incident report in the manually analyzed subset of the set of incident reports by:
labeling the at least one incident report in the manually analyzed subset as severe; and
labeling at least one additional incident report in the manually analyzed subset as not severe.
16. The system of claim 15 , wherein the performing module performs the action by at least one of:
forwarding the incident report labeled as severe to an analyst for manual analysis in response to labeling the incident report as severe; and
avoiding forwarding the additional incident report labeled as not severe to an analyst for manual analysis in response to labeling the additional incident report as not severe.
17. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
identify:
a set of incident reports that describe incidents that each involve at least one computing system and that comprise automatically collected information about the incidents; and
a manually analyzed subset of the set of incident reports that further comprise manually generated information about the incidents in addition to automatically collected information about the incidents;
assign at least one label from a set of labels to at least one incident report in the manually analyzed subset of the set of incident reports, wherein the set of labels was generated based on applying a machine learning model to the manually generated information;
derive, from the automatically collected information, a set of features that describe the set of incident reports wherein each feature in the set of features discriminates between differing types of incident reports;
plot each incident report in the set of incident reports on a graph based on at least one value for the incident report of at least one feature in the set of features;
in response to determining that the labeled incident report and the incident report comprise similar features from the set of features by determining that the labeled incident report and the incident report are plotted within a predetermined distance of each other in the graph,
propagate at least one label assigned to a labeled incident report in the manually analyzed subset of the set of incident reports to an incident report that is not in the manually analyzed subset and that comprises similar features from the set of features with the labeled incident report; and
perform an action related to the label on the incident report that is not in the manually analyzed subset in response to propagating the label to the incident report.
18. The non-transitory computer-readable medium of claim 17 , wherein the one or more computer-readable instructions cause the computing device to perform the action by forwarding the incident report to an analyst for manual analysis.
19. The non-transitory computer-readable medium of claim 17 , wherein performing the action comprises propagating at least one manually added note from the labeled incident report to the incident report.
20. The non-transitory computer-readable medium of claim 17 , wherein the incidents comprise security incidents and the set of incident reports comprises incident reports generated by security applications.