Platform for managing threat data
A platform for managing threat data integrates threat data from a variety of sources including internal threat data from instrumented compute instances associated with an enterprise network and threat data from one or more independent, external resources. Threat assessments are incrementally revised as this threat data is asynchronously received from various sources, and a threat intervention container is automatically created and presented to an investigator when a composite threat score for one or more of the compute instances meets a predetermined threshold.
1. A system comprising:
a plurality of compute instances coupled to an enterprise network;
a data lake storing an event stream from a number of sources of security data, the event stream including:
security events from one or more sensors on the plurality of compute instances coupled to the enterprise network;
cloud resource data from a cloud service supporting the plurality of compute instances coupled to the enterprise network; and
contextual data for activity by the plurality of compute instances from a third-party service; and
a threat management facility configured to perform the steps of:
updating a composite threat score based on the number of sources of security data as the event stream is asynchronously received from the number of sources of security data, wherein updating the composite threat score includes mapping the security events to an attack matrix enumerating malware strategies in a first dimension and malware techniques for each of the malware strategies in a second dimension, and calculating the composite threat score based on a pattern of traversal of the attack matrix by a chronology of the security events,
automatically creating an investigation container for interactive investigation of security risks when the composite threat score meets one or more predetermined criteria for initiating an investigation,
displaying the investigation container to a user, and
incrementally updating the investigation container with information from the number of sources of security data.
2. The system of claim 1 , wherein the security events include asynchronous data from the plurality of compute instances.
3. The system of claim 1 , wherein the security events include batch data from one or more of the plurality of compute instances.
4. The system of claim 1 , wherein the plurality of compute instances include at least one network device for the enterprise network.
5. The system of claim 1 , wherein the plurality of compute instances include at least one virtual computing device hosted on a virtualization platform.
6. The system of claim 1 , wherein the security events include threat detection data from a local security agent executing on one of the plurality of compute instances.
7. The system of claim 1 , wherein the contextual data includes geolocation data from the third-party service.
8. The system of claim 1 , wherein the data lake applies deduplication logic to remove one or more recurring detections from at least one of the one or more sensors.
9. The system of claim 1 , wherein the cloud service includes one or more of a web application, a cloud storage service, an electronic mail application, an authentication service, a zero trust network access resource, a cloud computing service, and a virtualization platform.
10. The system of claim 1 , wherein the cloud service includes a network monitor executing on a third-party firewall and securely coupled to the threat management facility.
11. The system of claim 1 , wherein the number of sources of security data include a monitor for a query interface to the data lake.
12. The system of claim 1 , wherein the threat management facility is configured to present a user interface associated with the investigation container in a display to the user.
13. The system of claim 1 , wherein the threat management facility is configured to transmit a link to a user interface associated with the investigation container in a message to the user.
14. The system of claim 1 , wherein the threat management facility is configured to display a user interface associated with the investigation container, the user interface including a control for requesting additional event data captured by a data recorder of one of the plurality of compute instances prior for a time window prior to creation of the investigation container.
15. The system of claim 1 , wherein the threat management facility is configured to display a user interface associated with the investigation container, the user interface including a control for adjusting a filter applied by one of the plurality of compute instances to a local event stream when selecting local security events to communicate on the event stream to the data lake.
16. The system of claim 1 , wherein the threat management facility is configured to display a user interface associated with the investigation container, the user interface including a tool for querying the data lake.
17. The system of claim 1 , wherein the threat management facility is configured to calculate the composite threat score by applying a machine learning algorithm to the pattern of traversal of the attack matrix to determine a likelihood of threat.
18. The system of claim 1 , wherein the composite threat score includes two or more scores based on two or more of the number of sources of security data, and wherein the one or more predetermined criteria for initiating the investigation includes a separate threshold for each of the two or more scores.
19. The system of claim 1 , wherein the threat management facility is configured to revise the composite threat score downward when a false positive is identified, and automatically close the investigation container when the composite threat score meets a second one or more predetermined criteria for terminating the investigation.
20. A method comprising:
with a data lake for an enterprise network, storing an event stream from a number of sources of security data, the event stream including:
security events from one or more sensors on a plurality of compute instances coupled to the enterprise network;
cloud resource data from a cloud service supporting the plurality of compute instances coupled to the enterprise network; and
contextual data for activity by the plurality of compute instances from a third-party service; and
with a threat management facility executing on one or more processors, performing the steps of:
updating a composite threat score based on the number of sources of security data as the event stream is asynchronously received from the number of sources of security data, wherein updating the composite threat score includes mapping the security events to an attack matrix enumerating malware strategies in a first dimension and malware techniques for each of the malware strategies in a second dimension, and calculating the composite threat score based on a pattern of traversal of the attack matrix by a chronology of the security events,
automatically creating an investigation container for interactive investigation of security risks when the composite threat score meets one or more predetermined criteria for initiating an investigation,
displaying the investigation container to a user, and
incrementally updating the investigation container with information from the number of sources of security data.