SYSTEMS AND METHODS FOR AUTOMATICALLY TUNING ONE OR MORE API POLLERS IN A CYBERSECURITY EVENT DETECTION AND RESPONSE SERVICE
A system, method, and computer-program product includes generating a polling task that is configured to retrieve raw event data of a subscriber that occurred during a target time span from a third-party security service, computing an optimal polling frequency control value for the polling task in response to generating the polling task, partitioning the target time span into a plurality of distinct sub-intervals of time based on the optimal polling frequency control value computed for the polling task, automatically transmitting, to an application programming interface endpoint of the third-party security service, a plurality of distinct network requests to optimally perform the polling task, receiving the raw event data of the subscriber from the third-party security service that occurred during the target time span in response to transmitting the plurality of distinct network request, and generating one or more security alerts in response to processing the raw event data.
1 . A computer-implemented method comprising:
generating a polling task that is configured to retrieve event data that occurred during a target time span from a target service;
computing a control value for the polling task in response to generating the polling task, wherein computing the control value includes:
computing a first candidate control value and a second candidate control value based in part on polling performance data sourced from a computer database, and
in response to assessing the first candidate control value against the second candidate control value, electing one of the first candidate control value and the second candidate control value as the control value for the polling task;
partitioning the target time span into a plurality of distinct sub-intervals of time based on the control value computed for the polling task;
automatically transmitting, to an application programming interface endpoint of the target service, a plurality of distinct network requests to perform the polling task, wherein each distinct network request corresponds to (i) a distinct sub-interval of time of the plurality of distinct sub-intervals of time and (ii) is configured to retrieve a respective portion of the event data that corresponds to the distinct sub-interval of time of the plurality of distinct sub-intervals of time; and
in response to transmitting the plurality of distinct network requests, receiving the event data from the target service that occurred during the target time span.
2 . The computer-implemented method according to claim 1 , further comprising:
generating one or more security alerts based on processing the event data received from the target service; and
executing a threat mitigation response that mitigates a security threat associated with the one or more security alerts by:
(i) surfacing the one or more security alerts in real-time to a user interface, and
(ii) generating one or more proposed threat remediation actions for the one or more security alerts that, when implemented, mitigates the security threat.
3 . The computer-implemented method according to claim 1 , further comprising:
generating one or more security alerts based on processing the event data received from the target service, wherein the one or more security alerts includes a compromised digital asset;
executing, in real-time, a threat mitigation response that mitigates a security threat associated with the compromised digital asset by:
automatically executing, in real-time, one or more automated threat remediation actions that adapts a configuration of a digital environment to resolve the security threat.
4 . The computer-implemented method according to claim 1 , wherein:
the polling performance data is associated with a plurality of historical polling steps, and
the first candidate control value is computed based in part on memory used during the plurality of historical polling steps.
5 . The computer-implemented method according to claim 1 , wherein:
the polling performance data is associated with a plurality of historical polling steps, and
the second candidate control value is computed based in part on an amount of time elapsed to complete the plurality of historical polling steps.
6 . The computer-implemented method according to claim 1 , wherein:
the first candidate control value specifies a first amount of time,
the second candidate control value specifies a second amount of time, and
the first candidate control value is elected as the control value for the polling task based on detecting that the first amount of time is less than the second amount of time.
7 . The computer-implemented method according to claim 1 , wherein:
the first candidate control value specifies a first amount of time,
the second candidate control value specifies a second amount of time, and
the second candidate control value is elected as the control value for the polling task based on detecting that the second amount of time is less than the first amount of time.
8 . The computer-implemented method according to claim 1 , wherein:
assessing the first candidate control value against the second candidate control value includes detecting a minimum between the first candidate control value and the second candidate control value, and
the first candidate control value is elected as the control value for the polling task based on detecting that the first candidate control value is the minimum between the first candidate control value and the second candidate control value.
9 . The computer-implemented method according to claim 1 , wherein:
assessing the first candidate control value against the second candidate control value includes detecting a minimum between the first candidate control value and the second candidate control value, and
the second candidate control value is elected as the control value for the polling task based on detecting that the second candidate control value is the minimum between the first candidate control value and the second candidate control value.
10 . The computer-implemented method according to claim 1 , wherein partitioning the target time span into the plurality of distinct sub-intervals of time includes:
partitioning the target time span into a first sub-interval of time that corresponds to a first distinct portion of the target time span,
partitioning the target time span into a second sub-interval of time that corresponds to a second distinct portion of the target time span, wherein the second sub-interval of time does not overlap with the first sub-interval of time, and
partitioning the target time span into a third sub-interval of time that corresponds to a third distinct portion of the target time span, wherein the third sub-interval of time does not overlap with the first sub-interval of time nor the second sub-interval of time.
11 . The computer-implemented method according to claim 10 , further comprising:
automatically creating a first distinct network request that is configured to retrieve the respective portion of the event data that corresponds to the first distinct portion of the target time span;
automatically creating a second distinct network request that is configured to retrieve the respective portion of the event data that corresponds to the second distinct portion of the target time span; and
automatically creating a third distinct network request that is configured to retrieve the respective portion of the event data that corresponds to the third distinct portion of the target time span.
12 . The computer-implemented method according to claim 11 , wherein automatically transmitting the plurality of distinct network requests to the application programming interface endpoint of the target service includes:
electronically transmitting, over a computer network, the first distinct network request to the application programming interface endpoint of the target service to retrieve the respective portion of the event data that corresponds to the first distinct portion of the target time span,
electronically transmitting, over the computer network, the second distinct network request to the application programming interface endpoint of the target service to retrieve the respective portion of the event data that corresponds to the second distinct portion of the target time span, and
electronically transmitting, over the computer network, the third distinct network request to the application programming interface endpoint of the target service to retrieve the respective portion of the event data that corresponds to the third distinct portion of the target time span.
13 . The computer-implemented method according to claim 12 , wherein receiving the event data from the target service includes:
receiving, from the target service, the respective portion of the event data that corresponds to the first distinct portion of the target time span in response to transmitting the first distinct network request to the application programming interface endpoint of the target service,
receiving, from the target service, the respective portion of the event data that corresponds to the second distinct portion of the target time span in response to transmitting the second distinct network request to the application programming interface endpoint of the target service, and
receiving, from the target service, the respective portion of the event data that corresponds to the third distinct portion of the target time span in response to transmitting the third distinct network request to the application programming interface endpoint of the target service.
14 . A method comprising:
generating a polling task that is configured to retrieve event data that occurred during a target time span from a target service;
computing a control value for the polling task in response to generating the polling task, wherein computing the control value includes:
computing a first candidate control value and a second candidate control value based in part on polling performance data sourced from a computer database, and
in response to assessing the first candidate control value against the second candidate control value, electing one of the first candidate control value and the second candidate control value as the control value for the polling task;
partitioning the target time span into a plurality of distinct sub-intervals of time based on the control value computed for the polling task;
automatically transmitting, to an application programming interface endpoint of the target service, a plurality of distinct network requests to perform the polling task, wherein each distinct network request corresponds to (i) a distinct sub-interval of time of the plurality of distinct sub-intervals of time and (ii) is configured to retrieve a respective portion of the event data that corresponds to the distinct sub-interval of time of the plurality of distinct sub-intervals of time; and
in response to transmitting the plurality of distinct network requests, receiving the event data from the target service that occurred during the target time span.
15 . The method according to claim 14 , wherein the polling performance data includes:
a first amount of memory that was used during a first historical polling step,
a second amount of memory that was used during a second historical polling step,
a third amount of memory that was used during a third historical polling step,
a fourth amount of memory that was used during a fourth historical polling step, and
a fifth amount of memory that was used during a fifth historical polling step.
16 . The method according to claim 15 , wherein the polling performance data further includes:
a first polling step execution time value that corresponds to a total time duration required to complete the first historical polling step,
a second polling step execution time value that corresponds to the total time duration required to complete the second historical polling step,
a third polling step execution time value that corresponds to the total time duration required to complete the third historical polling step,
a fourth polling step execution time value that corresponds to the total time duration required to complete the fourth historical polling step, and
a fifth polling step execution time value that corresponds to the total time duration required to complete the fifth historical polling step.
17 . The method according to claim 14 , wherein:
assessing the first candidate control value against the second candidate control value includes detecting a minimum between the first candidate control value and the second candidate control value, and
the first candidate control value is elected as the control value for the polling task based on detecting that the first candidate control value is the minimum between the first candidate control value and the second candidate control value.
18 . The method according to claim 14 , wherein:
assessing the first candidate control value against the second candidate control value includes detecting a minimum between the first candidate control value and the second candidate control value, and
the second candidate control value is elected as the control value for the polling task based on detecting that the second candidate control value is the minimum between the first candidate control value and the second candidate control value.
19 . The method according to claim 14 , further comprising:
generating one or more security alerts based on processing the event data received from the target service.
20 . The method according to claim 14 , wherein:
the polling performance data is associated with a plurality of historical polling steps,
the first candidate control value is computed based in part on an amount memory used during the plurality of historical polling steps, and
the second candidate control value is computed based in part on an amount of time used to complete the plurality of historical polling steps.