Managing analytic results in a cybersecurity system
In a computer-implemented method for managing analytic results in a cybersecurity system, data representing a plurality of events are accessed, where the plurality of events include machine data generated by entities that are part of or that interact with a computer network. A cybersecurity analytic of a cybersecurity application is applied to the data to produce analytic results, wherein the cybersecurity analytic is to detect a cybersecurity-related anomaly or threat. A performance of the cybersecurity analytic is then evaluated by applying the analytic results to a specified performance criterion. A corrective action for the cybersecurity analytic is then determined, based on a result of evaluating the performance of the cybersecurity analytic. Zero or more anomaly or threat detections by the cybersecurity analytic are then incorporated into an output of the cybersecurity application, based on the determined corrective action.
1. A computer-implemented method comprising:
accessing, by a first computer system, data representing a plurality of events, the plurality of events including machine data generated by a plurality of entities that are part of or that interact with a computer network;
applying, by the first computer system, a cybersecurity analytic of a cybersecurity application to the data to produce a plurality of analytic results, wherein the cybersecurity analytic is to detect a cybersecurity-related anomaly or threat;
evaluating, by the first computer system, a performance of the cybersecurity analytic by at least determining whether a number of anomaly or threat detections produced by the cybersecurity analytic for a specified time interval exceeds a first threshold or falls below a second threshold different from the first threshold;
determining, by the first computer system, a corrective action for the cybersecurity analytic, based on a result of the evaluating the performance of the cybersecurity analytic;
incorporating, by the first computer system, zero or more anomaly or threat detections by the cybersecurity analytic into an output of the cybersecurity application, based on the determined corrective action, wherein the output is to be sent to an external user computer system; and
providing the output of the cybersecurity application to the external user computer system.
2. The method as recited in claim 1 , wherein the corrective action comprises throttling operability of the cybersecurity analytic by a prescribed factor to prevent some of the anomaly or threat detections produced by the cybersecurity analytic from being included in the output of the cybersecurity application when the number of anomaly or threat detections exceeds the second threshold and falls below the first threshold.
3. The method as recited in claim 1 , wherein the corrective action comprises preventing a specified percentage or portion of all anomaly or threat detections produced by the cybersecurity analytic for a particular time interval from being included in the output of the cybersecurity application when the number of anomaly or threat detections exceeds the second threshold and falls below the first threshold.
4. The method as recited in claim 1 , wherein the corrective action comprises disabling the cybersecurity analytic when the number of anomaly or threat detections exceeds the first threshold.
5. The method as recited in claim 1 , wherein the corrective action comprises throttling operability of the cybersecurity analytic by (i) a first prescribed factor to reduce the number of anomaly or threat detections produced by the cybersecurity analytic from being included in the output of the cybersecurity application when the number of anomaly or threat detections exceeds a third threshold greater than the second threshold and less than the first threshold and (ii) a second prescribed factor less than the first prescribed factor to reduce, by a lesser amount, the number of anomaly or threat detections produced by the cybersecurity analytic when the number of anomaly or threat detections falls below the third threshold and is greater than the second threshold.
6. The method as recited in claim 1 , wherein the evaluating comprises applying the number of anomaly or threat detections output by the cybersecurity analytic for a specified time interval to a range of thresholds including the first threshold to denote a number of anomaly or threat detections beyond a normal or expected range that could make the cybersecurity analytics unreliable and the second threshold to denote a number of anomaly or threat detections below the normal or expected range in which the cybersecurity analytics is ineffective.
7. The method as recited in claim 1 , wherein the evaluating comprises:
applying the number of anomaly or threat detections output by the cybersecurity analytic for a specified time interval to the first threshold; and
determining that the cybersecurity analytic is overfiring if the number of anomaly or threat detections output by the cybersecurity analytic for the specified time interval is above the first threshold.
8. The method as recited in claim 1 , wherein the evaluating comprises:
applying a number of anomaly or threat detections output by the cybersecurity analytic for a specified time interval to the second threshold; and
determining that the cybersecurity analytic is underperforming if the number of anomaly or threat detections output by the cybersecurity analytic for the specified time interval is below the second threshold.
9. The method as recited in claim 1 , wherein the evaluating comprises:
applying the number of anomaly or threat detections output by the cybersecurity analytic for a specified time interval to the first threshold and the second threshold; and
determining that the cybersecurity analytic is overfiring if the number of anomaly or threat detections output by the cybersecurity analytic for the specified time interval is above the first threshold, or that the cybersecurity analytic is underperforming if the number of anomaly or threat detections output by the cybersecurity analytic for the specified time interval is below the second threshold.
10. The method as recited in claim 1 , wherein the data representing the plurality of events are received by the first computer system from the external user computer system prior to the accessing of the data representing the plurality of events, and wherein the first computer system is a cloud-based computer system and the user computer system is an on-premises computer system.
11. The method as recited in claim 1 , wherein the evaluating comprises applying a random sampling of analytic results produced by the cybersecurity analytic to the first threshold and the second threshold.
12. The method as recited in claim 1 , wherein the evaluating is performed by a machine learning runtime.
13. The method as recited in claim 1 , further comprising:
executing the applying, the evaluating, the determining and the incorporating, for each of a plurality of cybersecurity analytics, including determining a separate corrective action for each of the plurality of cybersecurity analytics, wherein each of the plurality of cybersecurity analytics is to detect a different type of cybersecurity-related anomaly or threat.
14. A computer system comprising:
a processor; and
a non-transitory computer-readable medium having stored thereon instructions, execution of which by the processor causes the computer system to perform operations including:
accessing data representing a plurality of events, the plurality of events including machine data generated by a plurality of entities that are part of or that interact with a computer network;
applying a cybersecurity analytic of a cybersecurity application to the data to produce a plurality of analytic results, wherein the cybersecurity analytic is to detect a cybersecurity-related anomaly or threat;
evaluating a performance of the cybersecurity analytic by at least determining whether a number of anomaly or threat detections produced by the cybersecurity analytic for a specified time interval exceeds a first threshold or falls below a second threshold different from the first threshold;
determining a corrective action for the cybersecurity analytic, based on a result of the evaluating the performance of the cybersecurity analytic;
incorporating zero or more anomaly or threat detections by the cybersecurity analytic into an output of the cybersecurity application, based on the determined corrective action, wherein the output is to be sent to an external user computer system; and
providing the output of the cybersecurity application to the external user computer system.
15. The computer system as recited in claim 14 , wherein the corrective action comprises throttling operability of the cybersecurity analytic by a prescribed factor to prevent some of the anomaly or threat detections produced by the cybersecurity analytic from being included in the output of the cybersecurity application when the number of anomaly or threat detections exceeds the second threshold and falls below the first threshold.
16. The computer system as recited in claim 14 , wherein the corrective action comprises disabling the cybersecurity analytic when the number of anomaly or threat detections exceeds the first threshold.
17. The computer system as recited in claim 14 , wherein the corrective action comprises throttling operability of the cybersecurity analytic by (i) a first prescribed factor to reduce the number of anomaly or threat detections produced by the cybersecurity analytic from being included in the output of the cybersecurity application when the number of anomaly or threat detections exceeds a third threshold greater than the second threshold and less than the first threshold and (ii) a second prescribed factor less than the first prescribed factor to reduce, by a lesser amount, the number of anomaly or threat detections produced by the cybersecurity analytic when the number of anomaly or threat detections falls below the third threshold and is greater than the second threshold.
18. A non-transitory computer-readable medium having stored therein instructions, execution of which by one or more processors causes the one or more processors to perform operations including:
accessing data representing a plurality of events, the plurality of events including machine data generated by a plurality of entities that are part of or that interact with a computer network;
applying a cybersecurity analytic of a cybersecurity application to the data to produce a plurality of analytic results, wherein the cybersecurity analytic is to detect a cybersecurity-related anomaly or threat;
evaluating a performance of the cybersecurity analytic by at least determining whether a number of anomaly or threat detections produced by the cybersecurity analytic for a specified time interval exceeds a first threshold or falls below a second threshold different from the first threshold;
determining a corrective action for the cybersecurity analytic, based on a result of the evaluating the performance of the cybersecurity analytic;
incorporating zero or more anomaly or threat detections by the cybersecurity analytic into an output of the cybersecurity application, based on the determined corrective action, wherein the output is to be sent to an external user computer system; and
providing the output of the cybersecurity application to the external user computer system.
19. The non-transitory computer-readable medium as recited in claim 18 , wherein the corrective action comprises at least one of:
preventing some anomaly or threat detections produced by the cybersecurity analytic from being included in the output of the cybersecurity application when the number of anomaly or threat detections exceeds the second threshold and falls below the first threshold; or
disabling the cybersecurity analytic when the number of anomaly or threat detections exceeds the first threshold.
20. The non-transitory computer-readable medium as recited in claim 19 , wherein the evaluating comprises applying the number of anomaly or threat detections output by the cybersecurity analytic for the specified time interval to the second threshold and determining that the cybersecurity analytic is underperforming if the number of anomaly or threat detections output by the cybersecurity falls below the second threshold.