Real-time threat alert forensic analysis
Methods and systems for security monitoring and response include assigning an anomaly score to each of a plurality of event paths that are stored in a first memory. Events that are cold, events that are older than a threshold, and events that are not part of a top-k anomalous path are identified. The identified events are evicted from the first memory to a second memory. A threat associated with events in the first memory is identified. A security action is performed responsive to the identified threat.
1. A method for security monitoring and response, comprising:
assigning an anomaly score to each of a plurality of event paths that are stored in a first memory;
identifying events that are cold, events that are older than a threshold, and events that are not part of a top-k anomalous path;
evicting the identified events from the first memory to a second memory, wherein the first memory has a faster access time than the second memory;
identifying a threat associated with events in the first memory; and
performing a security action responsive to the identified threat.
2. The method of claim 1 , wherein the first memory is a random access memory and the second memory is disk-based memory.
3. The method of claim 1 , wherein identifying events that are cold comprises determining that a number of events that are generated by a given event is below a threshold.
4. The method of claim 1 , wherein identifying events that are old comprises moving events from a young generation pool to an old generation pool after a threshold time has passed.
5. The method of claim 1 , wherein identifying events that are not part of a top-k anomalous path includes adding a per-event anomaly score for each event in a path to generate an aggregated path anomaly score, wherein the top-k anomalous paths are determined as those k paths having the highest aggregated path anomaly scores.
6. The method of claim 1 , further comprising receiving a new event that is generated by an evicted event.
7. The method of claim 6 , further comprising:
reading an aggregated path anomaly score for the evicted event from the second memory;
adding a per-event anomaly score for the new event to the aggregated path anomaly score for the evicted event to generate a new path anomaly score; and
comparing the new path anomaly score to top-k aggregated path anomaly scores.
8. The method of claim 7 , further comprising evicting the new event, responsive to a determination that the new path anomaly score is lower than the top-k aggregated path anomaly scores.
9. The method of claim 7 , further comprising reading the evicted event from the second memory to the first memory, responsive to a determination that the new path anomaly score is higher than at least one of the top-k aggregated path anomaly scores.
10. A system for security monitoring and response, comprising:
a first memory that stores a plurality of event paths;
a second memory, wherein the first memory has a faster access time than the second memory;
an event aggregator configured to assign an anomaly score to each of the plurality of event paths;
a memory manager configured to identify events that are cold, events that are older than a threshold, and events that are not part of a top-k anomalous path, and to evict the identified events from the first memory to the second memory;
a threat detection system configured to identify a threat associated with events in the first memory; and
a security manager configured to perform a security action responsive to the identified threat.
11. The system of claim 10 , wherein the first memory is a random access memory and the second memory is a hard disk drive.
12. The system of claim 10 , wherein the memory manager is configured to identify events that are cold by determining that a number of events that are generated by a given event is below a threshold.
13. The system of claim 10 , wherein the memory manager is configured to identify events that are old by moving events from a young generation pool to an old generation pool after a threshold time has passed.
14. The system of claim 10 , wherein the memory manager is configured to identify events that are not part of a top-k anomalous path by adding a per-event anomaly score for each event in a path to generate an aggregated path anomaly score, wherein the top-k anomalous paths are determined as those k paths having the highest aggregated path anomaly scores.
15. The system of claim 10 , further comprising a network interface configured to receive a new event that is generated by an evicted event.
16. The system of claim 15 , wherein the event aggregator is further configured to read an aggregated path anomaly score for the evicted event from the second memory, and to add a per-event anomaly score for the new event to the aggregated path anomaly score for the evicted event to generate a new path anomaly score, and wherein the memory manager is further configured to compare the new path anomaly score to top-k aggregated path anomaly scores.
17. The system of claim 16 , wherein the memory manager is further configured to evict the new event, responsive to a determination that the new path anomaly score is lower than the top-k aggregated path anomaly scores.
18. The system of claim 16 , wherein the memory manager is further configured to read the evicted event from the second memory to the first memory, responsive to a determination that the new path anomaly score is higher than at least one of the top-k aggregated path anomaly scores.