Bifurcating security event processing
Disclosed herein are methods, systems, and processes to distribute and disperse search loads to optimize security event processing in cybersecurity computing environments. A search request that includes a domain specific language (DSL) query directed to a centralized search cluster by an event processing application is intercepted. The event processing application is inhibited from issuing the search request to the centralized search cluster if a structured or semi-structured document matches the DSL query.
1. A method, comprising:
maintaining a document index distinct from a document store implemented on a distributed search cluster, wherein:
the document store stores, as structured or semi-structured documents, user-defined automated tasks to be performed in response to security events, and
the document index stores at least some of the structured or semi-structured documents in the document store;
detecting occurrence of a security event;
intercepting a search request associated with the security event expressed as a domain specific language (DSL) query directed to the distributed search cluster;
without issuing the search request to the distributed search cluster or the document store:
determining that a structured or semi-structured document in the document index matches the DSL query in the search request; and
answering the search request using the structured or semi-structured document in the document.
2. The method of claim 1 , further comprising:
recording a search field and a data type in a metadata file;
mapping the search field to a key in the at least one structured or semi-structured document;
generating a mapping file;
identifying the key in the structured or semi-structured document;
parsing the structured or semi-structured document;
extracting a value of the key in the structured or semi-structured document; and
parsing the value or converting the value to the data type indicated in the metadata file.
3. The method of claim 1 , wherein
the security event is detected at a security server,
an event processing application executes on the security server, and
the security event causes a task to be performed as determined by execution of the DSL query.
4. The method of claim 3 , further comprising:
inhibiting the event processing application from forwarding the search request to the distributed search cluster.
5. The method of claim 1 , further comprising:
determining that the DSL query comprises multiple comparison statements;
comparing the structured or semi-structured document with each comparison statement in the DSL query; and
determining that the structured or semi-structured document matches the DSL query.
6. A non-transitory computer readable storage medium comprising program instructions executable to:
maintain a document index distinct from a document store implemented on a distributed search cluster, wherein:
the document store stores, as structured or semi-structured documents, user-defined automated tasks to be performed in response to security events, and
the document index stores at least some of the structured or semi-structured documents in the document store;
detect occurrence of a security event;
intercept a search request associated with the security event expressed as a domain specific language (DSL) query directed to the distributed search cluster;
without issuing the search request to the distributed search cluster or the document store:
determine that a structured or semi-structured document in the document index matches the DSL query in the search request; and
answer the search request using the structured or semi-structured document in the document index.
7. The method of claim 6 , further comprising:
recording a search field and a data type in a metadata file;
mapping the search field to a key in the at least one structured or semi-structured document;
generating a mapping file;
identifying the key in the structured or semi-structured document;
parsing the structured or semi-structured document;
extracting a value of the key in the structured or semi-structured document; and
parsing the value to converting the value to the data type indicated in the metadata file.
8. The method of claim 6 , wherein
the security event is detected at a security server,
an event processing application executes on the security server, and
the security event causes a task to be performed as determined by execution of the DSL query.
9. The method of claim 8 , further comprising:
inhibiting the event processing application from forwarding the search request to the distributed search cluster.
10. The method of claim 6 , further comprising:
determining that the DSL query comprises multiple comparison statements;
comparing the structured or semi-structured document with each comparison statement in the DSL query; and
determining that the structured or semi-structured document matches the DSL query.
11. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to:
maintain a document index distinct from a document store implemented on a distributed search cluster, wherein:
the document store stores, as structured or semi-structured documents, user-defined automated tasks to be performed in response to security events, and
the document index stores at least some of the structured or semi-structured documents in the document store;
detect occurrence of a security event;
intercept a search request associated with the security event expressed as a domain specific language (DSL) query directed to the distributed search cluster;
without issuing the search request to the distributed search cluster or the document store:
determine that a structured or semi-structured document in the document index matches the DSL query in the search request; and
answer the search request using the structured or semi-structured document in the document index.
12. The method of claim 11 , further comprising:
recording a search field and a data type in a metadata file;
mapping the search field to a key in the at least one structured or semi-structured document;
generating a mapping file;
identifying the key in the structured or semi-structured document;
parsing the structured or semi-structured document;
extracting a value of the key in the structured or semi-structured document; and
parsing the value to converting the value to the data type indicated in the metadata file.
13. The method of claim 11 , wherein
the security event is detected at a security server,
an event processing application executes on the security server, and
the security event causes a task to be performed as determined by execution of the DSL query.
14. The method of claim 13 , further comprising:
inhibiting the event processing application from forwarding the search request to the distributed search cluster.
15. The method of claim 11 , further comprising:
determining that the DSL query comprises multiple comparison statements;
comparing the structured or semi-structured document with each comparison statement in the DSL query; and
determining that the structured or semi-structured document matches the DSL query.