Machine learned alert classification system
Disclosed herein are systems, methods, and processes for a machine learned alert triaging classification (ATC) system that uses machine learning techniques to generate an alert triage classification model that can be trained and deployed in modern security operation centers to optimize alert triaging and cyber threat classification. A training dataset of classified records is obtained. Each classified record in the training dataset includes detection characteristics data of a set of machines and threat classification results produced by performing alert triage classification of detection messages associated with the set of machines. An ATC model is trained using the training dataset according to a machine learning technique. The training tunes the ATC model to classify, based on at least the detection characteristics data, a new detection message associated with a machine from the set of machines as a threat or as not a threat.
1 . A computer-implemented method, comprising:
executing program instructions stored on one or more memory devices of one or more computer devices using one or more hardware processors of the one or more computing devices, wherein the executing causes an alert triage classification system implemented by the one or more computer devices to perform:
monitoring a detection message queue associated with one or more detection source systems that protect a set of machines, wherein:
the detection message queue stores a plurality of detection messages associated with alerts generated for the set of machines received over a network,
the detection messages are received from one or more detection source systems operating on each of the set of machines, and
each detection messages comprises data associated with one or more machines protected by at least one detection source system and detection metadata associated with the at least one detection source system;
for individual ones of the detection messages:
accessing a merging key that is part of the detection message, wherein information in the merge key indicates whether the detection message is associated with an existing alert, and
based on the information in the merging key accessed, appending the detection message to the existing alert or generating another alert for the detection message;
using the detection messages to build a training dataset of a plurality of classified records;
training an alert triage classification model using the training dataset and according to a supervised machine learning technique, wherein the training tunes the alert triage classification model to classify, based on the training dataset, new detection messages associated with new alerts generated for the set of machines; and
after the training of the alert triage classification model:
using the is alert triage classification model to classify the new detection messages into a plurality of classifications, including a first classification indicating a first detection message as a threat and a second classification indicating a second detection message as not a threat;
comparing classification results of the alert triage classification model to security analyst classification results to determine an accuracy of the alert triage classification model; and
performing another training iteration of the alert triage classification model to increase the determined accuracy.
2 . The computer-implemented method of claim 1 , wherein
the alert triage classification model is trained based on a set of detection characteristics or parameters of the detection messages selected via a feature engineering interface.
3 . The computer-implemented method of claim 1 , wherein
classifications generated by the alert triage classification model are used to prioritize the new alerts when the new alerts are displayed on a security analyst console.
4 . The computer-implemented method of claim 1 , wherein
classifications generated by the alert triage classification model are associated with different threat classifications defined via a configuration interface.
5 . The computer-implemented method of claim 1 , wherein individual records in the training dataset comprise:
detection characteristics data associated with an alert, including (a) a hostname or internet protocol (IP) address of a machine identified by the alert, (b) a detection type of the alert, and (c) a severity level of the alert; and
a truth label indicating a threat classification result for the alert.
6 . The computer-implemented method of claim 5 , wherein
the truth label indicates a manual classification of the alert provided by a security analyst.
7 . The computer-implemented method of claim 1 , wherein
the detection messages in the detection message queue are used to generate different types of alerts selected from an alert database.
8 . The computer-implemented method of claim 1 , wherein
the merging key is used to group a set of related detection messages associated with a machine that are associated with:
a particular process,
one or more related processes,
a particular login,
a particular action or operation,
a particular source or cause, or
a particular type of threat.
9 . The computer-implemented method of claim 1 , wherein
the one or more detection source systems comprise one or more of:
a scan engine,
a vulnerability management (VM) system,
a security information and event management (SIEM) system,
a penetration testing system,
an application security testing system, and
a cloud security posture management (CSPM) system.
10 . The computer-implemented method of claim 1 , wherein
the alert triage classification model is deployed to a plurality of security operation centers (SOCs) operated by a managed detection and response (MDR) service provider, wherein the MDR service provider monitors computer networks of a plurality of organizations for cybersecurity threats.
11 . A system comprising:
one or more computer devices that include one or more memory devices and one or more hardware processors, wherein the one or more memory devices store program instructions that are executable by the one or more hardware processors to implement an alert triage classification system on the one or more computer devices, and cause the alert triage system to:
monitor a detection message queue associated with one or more detection source systems that protect a set of machines, wherein
the detection message queue stores a plurality of detection messages associated with alerts generated for the set of machines received over a network,
the detection messages are received from one or more detection source systems operating on each of the set of machines, and
each detection messages comprises data associated with one or more machines protected by at least one detection source system and detection metadata associated with the at least one detection source system;
for individual ones of the detection messages:
access a merging key that is part of the detection message, wherein information in the merge key indicates whether the detection message is associated with an existing alert, and
based on the information in the merging key accessed, append the detection message to the existing alert or generating another alert for the detection message;
use the detection messages to build a training dataset of a plurality of classified records;
train an alert triage classification model using the training dataset and according to a supervised machine learning technique, wherein the training tunes the alert triage classification model to classify, based on the training dataset, new detection messages associated with new alerts generated for the set of machines; and
after the alert triage classification model is trained:
use the is alert triage classification model to classify the new detection messages into a plurality of classifications, including a first classification indicating a first detection message as a threat and a second classification indicating a second detection message as not a threat;
compare classification results of the alert triage classification model to security analyst classification results to determine an accuracy of the alert triage classification model; and
perform another training iteration of the alert triage classification model to increase the determined accuracy.
12 . The system of claim 11 , wherein
the alert triage classification model is trained based on a set of detection characteristics or parameters of the detection messages selected via a feature engineering interface.
13 . The system of claim 11 , wherein
classifications generated by the alert triage classification model are used to prioritize the new alerts when the new alerts are displayed on a security analyst console.
14 . The system of claim 11 , wherein
classifications generated by the alert triage classification model are associated with different threat classifications defined via a configuration interface.
15 . The system of claim 11 , wherein
individual records in the training dataset comprise:
detection characteristics data associated with an alert, including (a) a hostname or internet protocol (IP) address of a machine identified by the alert, (b) a detection type of the alert, and (c) a severity level of the alert; and
a truth label indicating a threat classification result for the alert.
16 . The system of claim 15 , wherein
the truth label indicates a manual classification of the alert provided by a security analyst.
17 . The system of claim 11 , wherein
the detection messages in the detection message queue are used to generate different types of alerts selected from an alert database.
18 . The system of claim 11 , wherein
the merging key is used to group a set of related detection messages associated with a machine that are associated with:
a particular process,
one or more related processes,
a particular login,
a particular action or operation,
a particular source or cause, or
a particular type of threat.
19 . The system of claim 11 , wherein
the one or more detection source systems comprise one or more of:
a scan engine,
a vulnerability management (VM) system,
a security information and event management (SIEM) system,
a penetration testing system,
an application security testing system, and
a cloud security posture management (CSPM) system.
20 . The system of claim 11 , wherein
the alert triage classification model is deployed to a plurality of security operation centers (SOCs) operated by a managed detection and response (MDR) service provider, wherein the MDR service provider monitors computer networks of a plurality of organizations for cybersecurity threats.