Storage appliance and threat indicator query framework
Systems are described for capturing network traffic data and efficiently storing the data on solid state storage devices. The systems can include a capture process module, a storage management module, and a query module. The storage management module can include circuitry configured to hold an arbitrarily large number of solid state storage devices configured to appear to a host system as a single large solid state drive.
1. A workflow system for executing a query on network traffic data in connection with incident response, the system comprising:
agent software for execution at an endpoint, the software configurable for executing computer instructions for:
detecting a security event at the endpoint;
storing structured audit information relating to the detected event in a forensic file container;
transmitting the audit information relating to the event to an agent service as a forensic file container;
a processor configured at the agent service for managing multiple instances of the agent software executing at multiple endpoints and for receiving forensic file containers;
a processor configured for acquiring network traffic data;
a data store device for storing the network traffic data in a first packet capture file;
a processor configured for network traffic indexing by:
generating a subset of a dataset of the first packet capture file; and
generating a second packet capture file as an index based on the first packet capture file by selecting a subset of the network traffic data;
a processor configured for executing a query by:
receiving one or more parameters relating to information in the forensic file container;
identifying one or more parameters relating to network traffic data in the forensic file container; and
a processor configured for executing a query on the index for corresponding data in the first packet capture file, where the query parameter for the index file is based on the one or more parameters relating to network traffic data in the forensic file container; and
a processor configured for providing results responsive to a query on the acquired network traffic data through a secured web interface accessible only to clients with a current credential, an associated user certificate, or a secure token.
2. The system of claim 1 , wherein the query is based on a source internet protocol address or a destination internet protocol address match of a specified internet protocol address.
3. The system of claim 1 , wherein the query is based on only a source internet protocol address match of a specified internet protocol address.
4. The system of claim 1 , wherein the query is based on only a destination internet protocol address match of a specified internet protocol address.
5. The system of claim 1 , wherein the query is based on a source or destination port match of a specified port.
6. The system of claim 1 , wherein the query is based on only a source port match of a specified port.
7. The system of claim 1 , wherein the query is based on only a destination port match of a specified port.
8. The system of claim 1 , wherein the query is based on a packet time after a specified time.
9. The system of claim 1 , wherein the query is based on a packet time before a specified time.
10. The system of claim 1 , wherein the processor is further configured for defining and utilizing a common data structure for the packet capture files.
11. The system of claim 1 , wherein the event detection is based on the output of a registry audit module.
12. The system of claim 1 , wherein the event detection is based on the operation of a Bloom filter.
13. The system of claim 12 , wherein the event detection is based on finding anomalous activity at the endpoint.
14. The system of claim 1 , wherein the endpoint is an intrusion detection system.
15. A workflow method for executing a query on network traffic data in connection with incident response, the method comprising:
providing agent software for execution at an endpoint, the software configurable for executing computer instructions for:
detecting a security event at the endpoint;
storing structured audit information relating to the detected event in a forensic file container;
transmitting the audit information relating to the event to an agent service as a forensic file container;
executing an agent service at a processor configured for managing multiple instances of the agent software executing at multiple endpoints and for receiving forensic file containers;
acquiring network traffic data;
storing the network traffic data in a first packet capture file in a data store device;
indexing the network traffic by the processor by:
generating a subset of a dataset of the first packet capture file; and
generating a second packet capture file as an index based on the first packet capture file by selecting a subset of the network traffic data;
executing a query at the processor by:
receiving one or more parameters relating to information in the forensic file container;
identifying one or more parameters relating to network traffic data in the forensic file container;
executing a query on the index for corresponding data in the first packet capture file, where the query parameter for the index file is based on the one or more parameters relating to network traffic data in the forensic file container; and
a processor configured for providing results responsive to a query on the acquired network traffic data through a secured web interface accessible only to clients with a current credential, an associated user certificate, or a secure token.
16. The method of claim 15 , wherein the query is based on a source internet protocol address or a destination internet protocol address match of a specified internet protocol address.
17. The method of claim 15 , wherein the query is based on only a source internet protocol address match of a specified internet protocol address.
18. The method of claim 15 , wherein the query is based on only a destination internet protocol address match of a specified internet protocol address.
19. The method of claim 15 , wherein the query is based on a source or destination port match of a specified port.