SYSTEMS AND METHODS FOR AUTOMATED THREAT DETECTION USING RETROSPECTIVE AND PROSPECTIVE ASSESSMENT OF INDICATORS OF COMPROMISE

1 . A computer-implemented method comprising:

at a cybersecurity event detection and response service that is implemented by a network of distributed computers:

detecting, in real-time or near real-time, at least one indicator of compromise (IOC) included in a security artifact;

in response to detecting the at least one IOC, automatically generating, in real-time or near real-time, a threat hunt object that specifies (i) the at least one IOC, (ii) a temporal look-back parameter, and (iii) a plurality of distinct security devices eligible for IOC-based querying;

translating, using a large language model, the threat hunt object into a plurality of distinct threat hunt queries in response to providing the threat hunt object to the large language model, wherein:

each threat hunt query of the plurality of distinct threat hunt queries is written in a distinct query language required by a distinct security device of the plurality of distinct security devices, and

each threat hunt query of the plurality of distinct threat hunt queries includes the at least one IOC and specifies a retrospective query window based on the temporal look-back parameter;

in response to translating the threat hunt object into the plurality of distinct threat hunt queries:

simultaneously executing, using one or more pollers, the plurality of distinct threat hunt queries across the plurality of distinct security devices;

obtaining historical security event data from each security device of the plurality of distinct security devices in response to executing the plurality of distinct threat hunt queries; and

automatically constructing one or more IOC-based threat detection instructions using (a) the historical security event data obtained from each of the plurality of distinct security devices and (b) the at least one IOC;

assessing, in real-time or near real-time, new security events normalized by the cybersecurity event detection and response service against the one or more IOC-based threat detection instructions; and

automatically generating, in real-time or near real-time, at least one IOC-based security alert based on detecting that a respective new security event of the new security events satisfies one of the one or more IOC-based threat detection instructions.

2 . The computer-implemented method according to claim 1 , further comprising:

before assessing the new security events normalized by the cybersecurity event detection and response service:

initializing a counter for the at least one IOC;

while assessing the new security events normalized by the cybersecurity event detection and response service:

tracking, using the counter, a total number of times that the at least one IOC is detected in the new security events; and

after assessing the new security events normalized by the cybersecurity event detection and response service, performing at least one detection handling action based on the total number of times that the at least one IOC was detected in the new security events, wherein:

the at least one detection handling action includes terminating the one or more IOC-based threat detection instructions when the total number of times that the at least one IOC was detected in the new security events fails to satisfy a predetermined minimum IOC count threshold, and

the at least one detection handling action includes bypassing the termination of the one or more IOC-based threat detection instructions when the total number of times that the at least one IOC was detected in the new security events satisfies the predetermined minimum IOC count threshold.

3 . The computer-implemented method according to claim 1 , further comprising:

before assessing the new security events normalized by the cybersecurity event detection and response service:

initializing a counter for the at least one IOC, wherein the counter tracks a total number of times that the at least one IOC is detected in log data of all subscribers to the cybersecurity event detection and response service within a target time span;

at expiration of the target time span:

automatically assessing the total number of times that the at least one IOC was detected in the log data;

automatically extending, for a subsequent target time span, a duration that the one or more IOC-based threat detection instructions are active in the cybersecurity event detection and response service based on the assessment of the total number of times that the at least one IOC was detected in the log data; and

resetting the counter to an initial value to track a subsequent total number of times that the at least one IOC is detected in all computing environments of the all subscribers to the cybersecurity event detection and response service during the subsequent target time span; and

at expiration of the subsequent target time span:

automatically assessing the subsequent total number of times that the at least one IOC was detected during the subsequent target time span; and

automatically ceasing use of the one or more IOC-based threat detection instructions based on the assessment of the subsequent total number of times that the at least one IOC was detected during the subsequent target time span.

4 . The computer-implemented method according to claim 1 , further comprising:

in response to constructing the one or more IOC-based threat detection instructions:

assessing, in real-time or near real-time, the historical security event data obtained from each of the plurality of distinct security devices against the one or more IOC-based threat detection instructions;

generating, in real-time or near real-time, a plurality of distinct retrospective IOC-based security alerts based on detecting that at least a subset of the historical security event data obtained from the plurality of distinct security devices satisfies the one or more IOC-based threat

detection instructions; and

in response to generating the plurality of distinct retrospective IOC-based security alerts, automatically executing, in real-time or near real-time, one or more automated threat mitigation actions to mitigate or resolve a security threat associated with the plurality of distinct

retrospective IOC-based security alerts.

5 . The computer-implemented method according to claim 1 , wherein:

the historical security event data obtained from each security device of the plurality of distinct security devices includes:

a plurality of distinct sets of historical security events that correspond to a plurality of distinct subscribers to the cybersecurity event detection and response service, wherein each set of historical security events of the plurality of distinct sets of historical security events:

corresponds to a distinct subscriber of the plurality of distinct subscribers, and

includes all historical security events of the distinct subscriber that (1) occurred within the retrospective query window and (2) includes the at least one IOC, and

the computer-implemented method further includes:

in response to constructing the one or more IOC-based threat detection instructions:

assessing, in real-time or near real-time, the plurality of distinct sets of historical security events that correspond to the plurality of distinct subscribers against the one or more IOC-based threat detection instructions; and

generating, in real-time or near real-time, a plurality of distinct retrospective IOC-based security alerts for each distinct subscriber of the plurality of distinct subscribers based on the assessment of the plurality of distinct sets of

historical security events against the one or more IOC-based threat detection instructions.

6 . The computer-implemented method according to claim 1 , further comprising:

receiving, over a computer network, a third-party threat intelligence data feed that includes threat intelligence data comprising a plurality of distinct candidate IOCs, wherein:

the third-party threat intelligence data feed corresponds to the security artifact, and

detecting the at least one IOC includes:

assessing the plurality of distinct candidate IOCs to identify at least one candidate IOC of the plurality of distinct candidate IOCs that was not previously recognized by the cybersecurity event detection and response service as a known IOC, and

designating the at least one candidate IOC as the at least one IOC.

7 . The computer-implemented method according to claim 6 , wherein:

the plurality of distinct threat hunt queries, when executed across the plurality of distinct security devices, automatically performs a sweep of all computing environments of all subscribers to the cybersecurity event detection and response service to identify a plurality of historical security events that (1) occurred in the computing environments of the subscribers, (2) occurred within the retrospective query window, and (3) are associated with the at least one IOC, wherein each historical security event of the plurality of historical security events includes the at least one IOC, and

the computer-implemented method further includes:

in response to identifying the plurality of historical security events, generating a plurality of distinct retrospective IOC-based security alerts based in part on the plurality of historical security events identified from executing the plurality of distinct threat hunt queries; and

displaying the plurality of distinct retrospective IOC-based security alerts on a graphical user interface.

8 . The computer-implemented method according to claim 1 , wherein:

the security artifact corresponds to a security incident detected in a compromised computing environment of a target subscriber to the cybersecurity event detection and response service,

the plurality of distinct threat hunt queries, when executed across the plurality of distinct security devices, automatically performs a retrospective cross-environment scanning operation across computing environments of a plurality of additional subscribers to identify a plurality of historical security events associated with the at least one IOC, wherein each historical security event of the plurality of historical security events:

occurred at one of the computing environments of the plurality of additional subscribers,

occurred within the retrospective query window, and

includes event metadata specifying the at least one IOC, and

the computer-implementing method further includes:

automatically generating a plurality of distinct retrospective IOC-based security alerts based on the plurality of historical security events identified during the retrospective cross-environment scanning operation;

automatically routing the plurality of distinct retrospective IOC-based security alerts to a security alert queue; and

executing, in response to routing the plurality of distinct retrospective IOC-based security alerts to the security alert queue, one or more threat mitigation actions to mitigate or resolve a security threat associated with the plurality of distinct retrospective IOC-based security alerts.

9 . The computer-implemented method according to claim 8 , wherein:

the plurality of additional subscribers are different from the target subscriber, and

before execution of the retrospective cross-environment scanning operation, the plurality of historical security events identified during the retrospective cross-environment scanning operation were determined to be non-malicious by the cybersecurity event detection and response service.

10 . The computer-implemented method according to claim 8 , wherein:

the plurality of additional subscribers are different from the target subscriber, and

before execution of the retrospective cross-environment scanning operation, the

cybersecurity event detection and response service did not detect the security threat associated with the plurality of historical security events identified during the retrospective cross-environment scanning operation.

11 . The computer-implemented method according to claim 1 , further comprising:

automatically commencing, at a predetermined interval, a sequence of operations to reduce a time to detect and remediate future security threats associated with the at least one IOC, wherein at least one iteration of the sequence of operations includes:

automatically polling, using the one or more pollers, the plurality of distinct security devices to obtain new security event data;

in response to receiving the new security event data, assessing, in real-time or near real-time, a normalized representation of the new security event data against the one or more IOC-based threat detection instructions;

automatically generating one or more additional IOC-based security alerts based on the normalized representation of the new security event data satisfying the one or more IOC-based threat detection instructions; and

executing, in real-time or near real-time, one or more threat mitigation actions to mitigate or resolve a security threat associated with each of the one or more additional IOC-based security alerts.

12 . The computer-implemented method according to claim 1 , wherein:

the at least one IOC included in the threat hunt object includes:

a first plurality of IOCs that correspond to a first class of IOCs,

a second plurality of IOCs that correspond to a second class of IOCs, and

a third plurality of IOCs that correspond to a third class of IOCs,

the plurality of distinct security devices included in the threat hunt object includes:

a first identifier that corresponds to a first distinct endpoint detection and response service,

a second identifier that corresponds to a second distinct endpoint detection and response service, and

a third identifier that corresponds to a third distinct endpoint detection and response service, and

translating the threat hunt object into the plurality of distinct threat hunt queries includes:

generating a first distinct threat hunt query that is written in the distinct query language required by the first distinct endpoint detection and response service, wherein the first distinct threat hunt query includes:

the first plurality of IOCs, the second plurality of IOCs, and the third plurality of IOCs,

the retrospective query window, and

an application programming interface (API) endpoint for the first distinct endpoint detection and response service,

generating a second distinct threat hunt query that is written in the distinct query language required by the second distinct endpoint detection and response service, wherein the second distinct threat hunt query includes:

the first plurality of IOCs, the second plurality of IOCs, and the third plurality of IOCs,

the retrospective query window, and

an API endpoint for the second distinct endpoint detection and response service, and

generating a third distinct threat hunt query that is written in the distinct query language required by the third distinct endpoint detection and response service, wherein the third distinct threat hunt query includes:

the first plurality of IOCs, the second plurality of IOCs, and the third plurality of IOCs,

the retrospective query window, and

an API endpoint for the third distinct endpoint detection and response service.

13 . The computer-implemented method according to claim 12 , wherein:

simultaneously executing the plurality of distinct threat hunt queries includes:

transmitting, using the one or more pollers, a first API request that includes the first distinct threat hunt query to the API endpoint that corresponds to the first distinct endpoint detection and response service,

transmitting, using the one or more pollers, a second API request that includes the second distinct threat hunt query to the API endpoint that corresponds to the second distinct endpoint detection and response service, and

transmitting, using the one or more pollers, a third API request that includes the third distinct threat hunt query to the API endpoint that corresponds to the third distinct endpoint detection and response service, and

obtaining the historical security event data from each security device of the plurality of distinct security devices includes:

in response transmitting the first API request to the API endpoint of the first distinct endpoint detection and response service, receiving, from the first distinct endpoint detection and response service, a first plurality of historical security events that:

were detected in one or more computing environments of one or more subscribers monitored by the first distinct endpoint detection and response service,

occurred within the retrospective query window, and

are associated with at least one of the first plurality of IOCs, the second plurality of IOCs, and the third plurality of IOCs,

in response transmitting the second API request to the API endpoint of the second distinct endpoint detection and response service, receiving, from the second distinct endpoint detection and response service, a second plurality of historical security events that:

were detected in the one or more computing environments of the one or more subscribers monitored by the second distinct endpoint detection and response service,

occurred within the retrospective query window, and

are associated with at least one of the first plurality of IOCs, the second plurality of IOCs, and the third plurality of IOCs, and

in response transmitting the third API request to the API endpoint of the third distinct endpoint detection and response service, receiving, from the third distinct endpoint detection and response service, a third plurality of historical security events that:

were detected in the one or more computing environments of the one or more subscribers monitored by the third distinct endpoint detection and response service,

occurred within the retrospective query window, and

are associated with at least one of the first plurality of IOCs, the second plurality of IOCs, and the third plurality of IOCs, wherein the historical security event data includes the first plurality of historical security events, the second plurality of historical security events, and the third plurality of historical security events.

14 . The computer-implemented method according to claim 1 , further comprising:

in response to providing the threat hunt object to the large language model, generating, using the large language model, a single database query based on the threat hunt object, wherein:

the single database query is executable against a target database,

the single database query includes the at least one IOC and the retrospective query window, and

the target database stores log data obtained from each of a plurality of distinct security services;

in response to the large language model generating the single database query, executing the single database query against the target database;

in response to executing the single database query, retrieving, from the target database, a corpus of logs that satisfy the single database query, wherein:

a first subset of the corpus of logs includes a first plurality of logs generated by a first distinct security service of the plurality of distinct security services,

a second subset of the corpus of logs includes a second plurality of logs generated by a second distinct security service of the plurality of distinct security services, and

a third subset of the corpus of logs includes a third plurality of logs generated by a third distinct security service of the plurality of distinct security services; and

generating a plurality of distinct retrospective IOC-based security alerts in response to assessing the corpus of logs retrieved from the target database against the one or more IOC-based threat detection instructions.

15 . The computer-implementing method according to claim 1 , wherein:

the at least one IOC includes a first plurality of IOCs that correspond to a first class of IOCs, a second plurality of IOCs that correspond to a second class of IOCs, and a third plurality of IOCs that correspond to a third class of IOCs, and

while generating the plurality of distinct threat hunt queries using the large language model:

detecting, by the large language model, that including the first plurality of IOCs, the second plurality of IOCs, and the third plurality of IOCs in a single threat hunt query would cause the single threat hunt query to exceed a maximum query size of the distinct security device to which the single threat hunt query corresponds; and

in response to the large language model detecting that the single threat hunt query would exceed the maximum query size, generating multiple threat hunt queries for the distinct security device to which the single threat hunt query corresponds, wherein:

a first threat hunt query of the multiple threat hunt queries includes the first plurality of IOCs and excludes the second plurality of IOCs and the third plurality of IOCs,

a second threat hunt query of the multiple threat hunt queries includes the second plurality of IOCs and excludes the first plurality of IOCs and the third plurality of IOCs, and

a third threat hunt query of the multiple threat hunt queries includes the third plurality of IOCs and excludes the first plurality of IOCs and the second plurality of IOCs, wherein:

the plurality of distinct threat hunt queries includes the first threat hunt query, the second threat hunt query, and the third threat hunt query, and

the first threat hunt query, the second threat hunt query, and the third threat hunt query collectively represent a logical equivalent of the single threat hunt query.

16 . The computer-implemented method according to claim 1 , wherein:

generating a respective threat hunt query of the plurality of distinct threat hunt queries includes:

obtaining, using the large language model, a plurality query components that define a query syntax of the distinct query language required by the distinct security device to which the respective threat hunt query corresponds,

determining, using the large language model, an IOC class of the at least one IOC;

selecting, using the large language model, a respective field identifier from the plurality query components that corresponds to the IOC class; and

encoding, using the large language model, the respective threat hunt query by inserting the at least one IOC after the respective field identifier in accordance with the query syntax.

17 . The computer-implemented method according to claim 1 , further comprising:

in response to executing the plurality of distinct threat hunt queries, generating query findings data using the historical security event data obtained from each security device of the plurality of distinct security devices, wherein the query findings data includes:

a total number of historical security events identified by the plurality of distinct threat hunt queries,

a total number of retrospective IOC-based security alerts generated based on the historical security event data obtained from each security device of the plurality of distinct security devices, and

one or more query execution errors associated with at least one of the plurality of distinct threat hunt queries;

automatically generating training data for the large language model using the query findings data; and

training the large language model using the training data to improve a translation of a subsequent threat hunt object into a plurality of subsequent threat hunt queries.

18 . The computer-implemented method according to claim 1 , wherein:

the one of the one or more IOC-based threat detection instructions specifies:

generating a subject IOC-based security alert when a subject new security event includes at least one piece of event metadata equivalent to the at least one IOC, and

the one of the one or more IOC-based threat detection instructions generated the at least one IOC-based security alert for the respective new security event based on the one of the one or more IOC-based threat detection instructions detecting that a respective piece of event metadata included in the respective new security event is equivalent to the at least one IOC.

19 . The computer-implemented method according to claim 1 , wherein:

constructing a respective IOC-based threat detection instruction of the one or more IOC-based threat detection instructions includes:

providing the historical security event data obtained from each of the plurality of distinct security devices to the large language model;

assessing, using the large language model, the historical security event data obtained from each of the plurality of distinct security devices to detect a malicious behavior pattern associated with the at least one IOC;

encoding, using the large language model, the respective IOC-based threat detection instruction to specify a detection condition that is satisfied when a subject new security event includes event metadata that (1) matches the malicious behavior pattern and (2) includes the at least one IOC, and

the respective IOC-based threat detection generated the at least one IOC-based security alert for the respective new security event based on detecting that the respective new security event satisfies the detection condition of the respective IOC-based threat detection instruction.

20 . The computer-implemented method according to claim 1 , further comprising:

tracking, in a computer database, a distinct query execution state for each of the plurality of distinct threat hunt queries, wherein the distinct query execution state indicates whether a respective threat hunt query has not been executed, is partially executed, or has completed execution;

detecting one or more system restarts or one or more transient failures during execution of the plurality of distinct threat hunt queries; and

in response to detecting the one or more system restarts or the one or more transient failures:

identifying, using the computer database, a first subset of the plurality of distinct threat hunt queries that are partially completed, a second subset of the plurality of distinct threat hunt queries that have completed execution, and a third subset of the plurality of distinct threat hunt queries that have not been executed;

in response to identifying the first subset of the plurality of distinct threat hunt queries, re-executing each distinct threat hunt query included in the first subset;

in response to identifying the second subset of the plurality of distinct threat hunt queries, bypassing re-execution of each distinct threat hunt query included in the second subset; and

in response to identifying the third subset of the plurality of distinct threat hunt queries, commencing asynchronous execution of each distinct threat hunt query included in the third subset.