Generative artificial intelligence-based multi-stage composite event processing
A data platform monitors a compute environment by performing multi-stage heuristic analysis of event data representing a plurality of events occurring within the environment. The platform utilizes multiple event analyzers, each configured according to a distinct analysis heuristic, to evaluate different subsets of the event data and generate corresponding output signals. A higher-level event analyzer applies a further heuristic to the multiple output signals to generate a composite alert signal, indicating whether the combination of analyzed events collectively represents a security intrusion or other anomalous condition of sufficient severity to warrant alerting. Based on the composite alert signal, the platform performs an alert-based operation, such as generating a user-facing alert, initiating an automated mitigation, or updating a contextual model of system behavior. By combining the analytical outputs of heterogeneous heuristics, the disclosed architecture enhances the accuracy and contextual relevance of automated intrusion detection within complex computing environments.
1 . A method comprising:
accessing, by a data platform monitoring a compute environment, event data representative of a plurality of events that occur within the compute environment;
analyzing, by the data platform using a first event analyzer that operates in accordance with a first analysis heuristic, a first set of one or more events included in the plurality of events to generate a first output signal;
analyzing, by the data platform using a second event analyzer that operates in accordance with a second analysis heuristic, a second set of one or more events included in the plurality of events to generate a second output signal;
analyzing, by the data platform using a third event analyzer that operates in accordance with a third analysis heuristic, the first and second output signals to generate a composite alert signal indicative of whether the first set of one or more events and the second set of one or more events are together indicative of one or more intrusions that are severe enough to be alerted on by the data platform; and
performing, by the data platform based on the composite alert signal, an alert-based operation with respect to the plurality of events.
2 . The method of claim 1 , wherein:
the first output signal is indicative of whether the first set of one or more events are together indicative of one or more intrusions that are severe enough to be alerted on by the data platform, and
the second output signal is indicative of whether the first second of one or more events are together indicative of one or more intrusions that are severe enough to be alerted on by the data platform.
3 . The method of claim 1 , wherein:
the first analysis heuristic comprises a first machine learning model,
the second analysis heuristic comprises a second machine learning model, and
the third analysis heuristic comprises a third machine learning model.
4 . The method of claim 3 , wherein one or more of the first, second, or third trained machine learning models comprises one or more large language models configured to perform generative artificial intelligence operations.
5 . The method of claim 3 , wherein one or more of the first, second, or third trained machine learning models are configured to be trained on training sets generated by generative artificial intelligence.
6 . The method of claim 1 , wherein the first analysis heuristic is configured in accordance with a first common characteristic of the first set of one or more events, the second analysis heuristic is configured in accordance with a second common characteristic of the second set of one or events different from the first common characteristic, and the third analysis heuristic is configured in accordance with a third common characteristic of the third set of one or events different from the first and second common characteristics.
7 . The method of claim 1 , further comprising:
analyzing, by the data platform using a fourth event analyzer that operates in accordance with a fourth analysis heuristic, the composite alert signal to generate contextual information associated with the composite alert signal.
8 . The method of claim 7 , wherein the performing the alert-based operation comprises presenting the contextual information together with information associated with an alert associated with the composite alert signal within a user interface.
9 . The method of claim 7 , wherein the contextual information includes one or more of a recommended remedial action associated with the composite alert signal, a feature associated with the composite alert signal, a significance level associated with the feature, or a narrative description associated with the composite alert signal.
10 . The method of claim 7 , wherein the fourth analysis heuristic comprises a large language model configured to perform generative artificial intelligence operations.
11 . The method of claim 1 , wherein the performing the alert-based operation comprises presenting a security alert within a user interface based on a characteristic associated with the composite alert signal satisfying one or more predetermined criteria.
12 . The method of claim 1 , wherein the performing the alert-based operation comprises abstaining from presenting a security alert within a user interface based on a characteristic associated with the composite alert signal failing to satisfy one or more predetermined criteria.
13 . The method of claim 1 , wherein the performing the alert-based operation further comprises presenting a plurality of selectable items each associated with a different facet of the composite alert signal.
14 . The method of claim 1 , wherein:
the first set of one or more events is initiated within the compute environment by a first entity, and the second set of one or more events is initiated within the compute environment by a second entity.
15 . The method of claim 1 , wherein the event data is collected using one or more agents deployed within the computing environment.
16 . The method of claim 1 , wherein the event data is collected using one or more agentless configurations.
17 . A computer program product embodied in a non-transitory computer-readable storage medium and comprising computer instructions for a data platform to perform a process comprising:
accessing event data representative of a plurality of events that occur within a compute environment;
analyzing, using a first event analyzer that operates in accordance with a first analysis heuristic, a first set of one or more events included in the plurality of events to generate a first output signal;
analyzing, using a second event analyzer that operates in accordance with a second analysis heuristic, a second set of one or more events included in the plurality of events to generate a second output signal;
analyzing, using a third event analyzer that operates in accordance with a third analysis heuristic, the first and second output signals to generate a composite alert signal indicative of whether the first set of one or more events and the second set of one or more events are together indicative of one or more intrusions that are severe enough to be alerted on; and
performing, based on the composite alert signal, an alert-based operation with respect to the plurality of events.
18 . The computer program product of claim 17 , wherein:
the first output signal is indicative of whether the first set of one or more events are together indicative of one or more intrusions that are severe enough to be alerted on by the data platform, and
the second output signal is indicative of whether the first second of one or more events are together indicative of one or more intrusions that are severe enough to be alerted on by the data platform.
19 . The computer program product of claim 17 , wherein the process further comprises:
analyzing, by the data platform using a fourth event analyzer that operates in accordance with a fourth analysis heuristic, the composite alert signal to generate contextual information associated with the composite alert signal.
20 . A system comprising:
memory storing instructions; and
one or more processors communicatively coupled to the memory and configured to execute the instructions to perform a process comprising:
accessing event data representative of a plurality of events that occur within a compute environment;
analyzing, using a first event analyzer that operates in accordance with a first analysis heuristic, a first set of one or more events included in the plurality of events to generate a first output signal;
analyzing, using a second event analyzer that operates in accordance with a second analysis heuristic, a second set of one or more events included in the plurality of events to generate a second output signal;
analyzing, using a third event analyzer that operates in accordance with a third analysis heuristic, the first and second output signals to generate a composite alert signal indicative of whether the first set of one or more events and the second set of one or more events are together indicative of one or more intrusions that are severe enough to be alerted on; and
performing, based on the composite alert signal, an alert-based operation with respect to the plurality of events.