Security alert meta-analysis for identifying causally related evidence of cyberattacks
A security alert meta-analysis (SAMA) system is disclosed capable of identifying causally related evidence of a cyberattack in a computing environment. In embodiments, the system builds a security data graph from security alerts generated by other security monitoring services. The security data graph links related entities (e.g. users and resources) in the computing environment and the entities to their associated security alerts. Edges in the graph are filtered based on edge weights to identify sub-graphs that represent clusters of causally related evidence probative of attacks. The evidence clusters are presented to analysts to be investigated further. In embodiments, the meta-analysis process is implemented as periodic jobs executed on a cluster of worker nodes. Advantageously, the disclosed system is able to filter through large volumes of alerts to reduce false positives, and group related alerts, possibly from different monitoring services, so that they can be investigated together.
1 . A system, comprising:
one or more computing devices that implement a security alert meta-analysis (SAMA) system, configured to:
receive security data about a computing environment of networked computing resources that includes a first type of monitoring service that generates a first type of security alert about the computing environment of networked computing resources and a second type of monitoring service that generates a second type of security alert about the computing environment of networked computing resources,
wherein the first type of monitoring service and the second type of monitoring service independently collect security data from at least one or more data collecting agents or sensors operating in the computing environment, and
wherein the first type of security alert and the second type of security alert are generated after the first and second types of monitoring services independently perform a first-tier analysis of the independently collected security data, and the analysis identifies an event or condition in the computing environment;
perform a meta-analysis of the first and second types of security alerts generated by the first and second types of monitoring services, including to:
generate a security data graph from the security data, wherein the security data graph contains (a) a first set of vertices that represent entities observed in the security data, (b) a second set of vertices that represent the security alerts generated by the first and second types of monitoring services, (c) a first set of edges that represent actions or relationships among the entities, and (d) a second set of edges that indicate entities involved in the security alerts generated by the first and second types of monitoring services, wherein the entities include different types of resources, accounts, or observations in the computing environment; and
identify at least one sub-graph of the security data graph, where the sub-graph connects a group of causally related security alerts that evidence a cyberattack on the computing environment and entities associated with the causally related security alerts; and
generate, via a user interface, a report that indicates the group of causally related security alerts and associated entities as evidence of the cyberattack, wherein the report reduces a number of false positives in the security alerts generated by the first and second types of security monitoring services.
2 . The system of claim 1 , wherein:
the SAMA system is implemented as part of a network-accessible provider service that hosts client networks on behalf of a plurality of clients;
the security monitoring services are implemented as part of the network-accessible provider service and configured to generate the security data based at least in part on monitoring of the client networks; and
the SAMA system is configured to perform the meta-analysis on individual ones of the client networks using security data of individual ones of the client networks.
3 . The system of claim 1 , wherein: the report includes an attack chain view that organizes the causally related security alerts into a plurality of attack stages.
4 . The system of claim 1 , wherein the SAMA system is configured to:
repeatedly generate security data graphs of the computing environment for successive time periods in a distributed graph database; and
schedule a data analytics job to perform the meta-analysis on a most recent security data graph, wherein the data analytics job is executed by a cluster of compute nodes to process respective portions of the most recent security data graph in parallel.
5 . The system of claim 1 , wherein the SAMA system is configured to:
identify multiple sub-graphs of causally related security alerts and associated entities as multiple evidence clusters of cyberattacks during the meta-analysis of the security data graph;
calculate risk scores for individual ones of the multiple evidence clusters based at least in part on a likelihood or severity of respective evidenced cyberattacks; and
report, via the user interface, the multiple evidence clusters with the risk scores, wherein the evidence clusters are ranked according to the risk scores.
6 . The system of claim 1 , wherein vertices in the security data graph indicate entity types of the entities, the entity types including two or more of:
a client account,
a user account,
an Identity and Access Management (IAM) role,
a role session,
an Internet Protocol (IP) address,
a Domain Name System (DNS) domain, and
a geographic location.
7 . A method, comprising:
performing, by a security alert meta-analysis (SAMA) system implemented by one or more computing devices:
receiving security data about a computing environment that includes a first type of monitoring service that generates a first type of security alert about the computing environment of networked computing resources and a second type of monitoring service that generates a second type of security alert about the computing environment of networked computing resources,
wherein the first type of monitoring service and the second type of monitoring service independently collect security data from at least one or more data collecting agents or sensors operating in the computing environment, and
wherein the first type of security alert and the second type of security alert are generated after the first and second types of monitoring services independently perform a first-tier analysis of the independently collected security data, and the analysis identifies an event or condition in the computing environment;
performing a meta-analysis of the first and second types of security alerts generated by the first and second types of monitoring services, including:
generating a security data graph from the security data, wherein the security data graph contains (a) a first set of vertices that represent entities observed in the security data, (b) a second set of vertices that represent the security alerts generated by the first and second types of monitoring services, (c) a first set of edges that represent actions or relationships among the entities, and (d) a second set of edges that indicate entities involved in the security alerts generated by the first and second types of monitoring services, wherein the entities include different types of resources, accounts, or observations in the computing environments; and
identifying at least one sub-graph of the security data graph, where the sub-graph connects a group of causally related security alerts that evidence a cyberattack on the computing environment and entities associated with the causally related security alerts; and
generating, via a user interface, a report that indicates the group of causally related security alerts and associated entities as evidence of the cyberattack, wherein the report reduces a number of false positives in the security alerts generated by the first and second types of security monitoring services.
8 . The method of claim 7 , wherein:
the SAMA system is configured to assign individual client networks to different resource cells; and
an individual resource cell is configured to perform the meta-analysis on security data of an individual client network independently from other resource cells.
9 . The method of claim 8 , further comprising the SAMA system:
reporting, via the user interface, multiple evidence clusters of causally related security alerts and associated entities identified in the security data graph; and
responsive to user input received via the user interface, changing a sort order of the multiple evidence clusters on the user interface based on different attributes of the evidence clusters, the different attributes including two or more of:
a risk score associated with an evidence cluster,
a number of entities in the evidence cluster,
a number of security alerts in the evidence cluster,
a starting time of the evidence cluster, and
a duration of the evidence cluster.
10 . The method of claim 7 , further comprising the SAMA system:
storing the security data graph in a distributed graph database; and
scheduling a data analytics job to perform the meta-analysis on the security data graph, wherein the data analytics job is executed by a cluster of compute nodes to process respective portions of the security data graph in parallel.
11 . The method of claim 10 , further comprising the SAMA system:
monitoring execution of the data analytics job to determine job status information; and
reporting the job status information on a job management interface during the execution of the data analytics job.
12 . The method of claim 8 , further comprising the SAMA system:
assigning that individual security alerts in the group to respective stages of the cyberattack; and
displaying, via the user interface, an attack chain that indicates security alerts assigned to the respective stages of the cyberattack.
13 . The method of claim 8 , wherein:
the SAMA system is configured to repeat the meta-analysis to identify sub-graphs of causally-related security alerts in individual time periods; and
the method further comprises the SAMA system:
receiving user input indicating to monitor a particular sub-graph identified in a first time period for additional time periods;
monitoring the particular sub-graph according to the user input, wherein the monitoring augments the particular sub-graph with additional causally related security alerts observed in the additional time periods; and
reporting the particular sub-graph including the additional causally related security alerts via the user interface.
14 . The method of claim 8 , further comprising the SAMA system:
receiving a user query via the user interface specifying a graph search starting from one or more selected vertices in the sub-graph; and
responsive to the user query, updating the user interface to provide additional causally related security alerts and associated entities found based on the graph search.
15 . A method of claim 7 , wherein identifying the sub-graph comprises:
calculating edge weights for edges in the security data graph, wherein an edge weight for a given edge reflects odds of the given edge being due to an attack activity versus a benign activity;
filtering the edges in the security data graph using the edge weights and a weight threshold; and
identifying a group of connected vertices as the sub-graph, wherein the group is connected with edges that remain after the filtering.
16 . The method of claim 15 , wherein:
the edge weights are calculated based at least on historical graph statistics of previous security data graphs of the computing environment; and
the historical graph statistics are maintained by a statistical model and updated periodically to reflect changes in the computing environment.
17 . The method of claim 7 , wherein the report includes an attack chain view that organizes the causally related security alerts into a plurality of attack stages.
18 . One or more non-transitory computer-readable storage media storing program instructions that when executed on or across one or more processors implement a security alert meta-analysis (SAMA) system and cause the SAMA system to:
receive security data about a computing environment that includes a first type of monitoring service that generates a first type of security alert about the computing environment of networked computing resources and a second type of monitoring service that generates a second type of security alert about the computing environment of networked computing resources,
wherein the first type of monitoring service and the second type of monitoring service independently collect security data from at least one or more data collecting agents or sensors operating in the computing environment,
wherein the first type of security alert and the second type of security alert is generated after the first and second types of monitoring services independently perform a first-tier analysis of the independently collected security data, and the analysis identifies an event or condition in the computing environment;
perform a meta-analysis of the first and second types of security alerts generated by the first and second types of monitoring services, including to:
generate a security data graph from the security data, wherein the security data graph contains (a) a first set of vertices that represent entities observed in the security data, (b) a second set of vertices that represent the security alerts, (c) a first set of edges that represent actions or relationships among the entities, and (d) a second set of edges that indicate entities involved in the security alerts generated by the first and second types of monitoring services, wherein the entities include different types of resources, accounts, or observations in the computing environment; and
identify at least one sub-graph of the security data graph, where the sub-graph connects a group of causally related security alerts that evidence a cyberattack on the computing environment and entities associated with the causally related security alerts; and
generate, via a user interface, a report that indicates the group of causally related security alerts and associated entities as evidence of the cyberattack, wherein the report reduces a number of false positives in the security alerts generated by the first and second types of security monitoring services.
19 . The one or more non-transitory computer-readable storage media of claim 18 , wherein the report includes an attack chain view that organizes the causally related security alerts into a plurality of attack stages.
20 . The one or more non-transitory computer-readable storage media of claim 18 , wherein the program instructions when executed on or across one or more processors cause the SAMA system to:
store the security data graph in a distributed graph database; and
schedule a data analytics job to perform the meta-analysis on the security data graph, wherein the data analytics job is executed by a cluster of compute nodes to process respective portions of the security data graph in parallel.