System and method for determining network security threats
System and methods for determining network threats are disclosed. For each entity operating in a network being monitored for network security, an example method obtains an observed metric value for each metric that characterizes actions performed by the entity. Each observed metric value may be input into a machine learning model that is specific to the metric in order to determine an anomaly score for the observed metric value that represents how anomalous the observed metric value is relative to an expected metric value for the metric. A threat score may then be determined for each entity from the anomaly scores for each metric. A security threat presentation that identifies one or more high-scoring entities according to the threat scores may be generated and provided for display on a user device.
1. A computer-implemented method for determining network threats for a computer network, the method comprising:
obtaining, for each of a plurality of entities operating in a computer network being monitored for network security, a respective observed metric value for each of a plurality of metrics that each represent a respective behavior exhibited by the entity, wherein the plurality of entities comprise at least one or more users or one or more host computers;
for each of the plurality of metrics,
training, using information about typical entity behavior in the monitored network from various sources, a respective machine learning model that is specific to the metric and that models an expected metric value for the metric;
for each of the plurality of entities and for each of the plurality of metrics:
determining, using the trained machine learning model that is specific to the metric and an anomaly score for the observed metric value for the entity that represents how anomalous the observed metric value is relative to an expected metric value for the metric by:
determining a probability of the observed metric value occurring in the computer network being monitored using the trained machine learning model that is specific to the metric; and
determining the anomaly score from the determined probability of the observed metric value occurring in the computer network being monitored, the anomaly score indicating how anomalous the observed metric value for the entity is relative to an expected metric value for the specific metric
aggregating the anomaly scores from the machine learning models specific to the respective metrics, to generate a respective threat score for each entity; and
determining detecting potential network threats based on the threat scores of the entities.
2. The computer-implemented method of claim 1 , further comprising:
ranking the observed metric values from most anomalous to least anomalous using the anomaly scores for the observed metric values; and
providing the ranked observed metric values for display on a user device.
3. The computer-implemented method of claim 1 , wherein aggregating the anomaly scores from the machine learning models specific to the respective metrics to generate a respective threat score for each entity further comprises:
for each of the plurality of entities:
determining anomalous metric values associated directly with the entity;
determining anomalous metric values associated indirectly with the entity;
organizing the direct and indirect anomalous metric values into threat stages;
aggregating anomalous metric values per threat stage;
aggregating the anomalous metric values across threat stages; and
providing the aggregation of the anomalous metric values for all threat stages of the entity as the threat score for the entity.
4. The computer-implemented method of claim 3 , wherein determining anomalous metric values associated indirectly with the entity further comprises:
determining anomalous metric values associated with one or more entities that are linked with the entity through a link path.
5. The computer-implemented method of claim 1 , further comprising:
organizing observed metric values into behaviors with which the observed metric values are associated; and
organizing the behaviors into threat stages with which the behaviors are associated.
6. The computer-implemented method of claim 1 further comprising:
generating a security threat presentation that identifies one or more highest-scoring entities according to the threat scores; and
providing the security threat presentation for display on a user device.
7. The computer-implemented method of claim 6 , wherein a security threat presentation that identifies one or more highest-scoring entities according to the threat scores includes for each entity,
a threat score for the entity,
a time range for when the threat score is valid,
a threat score at each threat stage,
a case story that summarizes the threat score, and
a link to details regarding the threat score.
8. The computer-implemented method of claim 6 , wherein the security threat presentation that identifies the one or more highest-scoring entities according to the threat scores includes for each entity:
a threat score for the entity,
a case story that summarizes the threat score, and
details regarding the threat score.
9. The computer-implemented method of claim 7 , wherein the details regarding the threat score further comprises:
information about why the entity is a threat including at least one observed anomalous metric value of the entity or at least one anomalous metric value of a second entity linked to the entity through a direct or an indirect link path.
10. The computer-implemented method of claim 9 , wherein information about why the entity is a threat further includes:
a date when the anomalous metric value was observed;
an acting entity with which the anomalous metric value is associated;
a behavior with which the anomalous metric value is associated;
a stage with which the behavior is associated;
an indicator explaining the anomalous metric value;
information about a link path, linking the acting entity to the entity; and
an anomaly score.
11. A system comprising:
one or more computers; and
one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
obtaining, for each of a plurality of entities operating in a computer network being monitored for network security, a respective observed metric value for each of a plurality of metrics that each represent a respective behavior exhibited by the entity, wherein the plurality of entities comprise one or more users or host computers;
for each of the plurality of metrics,
training using information about typical entity behavior in the monitored network from various sources, a respective machine learning model that is specific to the metric and that models an expected metric value for the metric;
for each of the plurality of entities and for each of the plurality of metrics:
determining, using the trained machine learning model that is specific to the metric and, an anomaly score for the observed metric value that represents how anomalous the observed metric value for the entity is relative to an expected metric value for the metric by:
determining a probability of the observed metric value occurring in the computer network being monitored using the trained machine learning model that is specific to the metric; and
determining the anomaly score from the determined probability of the observed metric value occurring in the computer network being monitored, the anomaly score indicating how anomalous the observed metric value is relative to an expected metric value for the specific metric,
aggregating the anomaly scores from the machine learning models specific to the respective metrics to generate a respective threat score for each entity; and
detecting potential network threats based on the threat scores of the entities.
12. The system of claim 11 , wherein the operations further comprise:
ranking the observed metric values from most anomalous to least anomalous using the anomaly scores for the observed metric values; and
providing the ranked observed metric values for display on a user device.
13. The system of claim 11 , wherein aggregating the anomaly scores from the machine learning models specific to the respective metrics to generate the respective threat score for each entity further comprises:
for each of the plurality of entities:
determining anomalous metric values associated directly with the entity;
determining anomalous metric values associated indirectly with the entity;
organizing anomalous metric values per threat stage;
aggregating anomalous metric values across threat stages; and
providing the aggregation of the anomalous metric values for all threat stages of the entity as the threat score for the entity.
14. The system of claim 13 , wherein determining anomalous metric values associated indirectly with the entity further comprises:
determining anomalous metric values associated with one or more entities that are linked with the entity through a link path.
15. The system of claim 11 , wherein the operations further comprise:
organizing observed metric values into behaviors with which the observed metric values are associated; and
organizing the behaviors into threat stages with which the behaviors are associated.
16. The system of claim 11 , wherein the operations further comprise:
generating a security threat presentation that identifies one or more highest-scoring entities according to the threat scores; and
providing the security threat presentation for display on a user device.
17. The system of claim 16 , wherein a security threat presentation that identifies one or more highest-scoring entities according to the threat scores includes for each entity,
a threat score for the entity,
a time range for when the threat score is valid,
a threat score at each threat stage,
a case story that summarizes the threat score, and
a link to details regarding the threat score.
18. The system of claim 16 , wherein the security threat presentation that identifies the one or more highest-scoring entities according to the threat scores includes for each entity:
a threat score for the entity,
a case story that summarizes the threat score, and
details regarding the threat score.
19. The system of claim 17 , wherein the details regarding the threat score further comprises:
information about why the entity is a threat including at least one observed anomalous metric value of the entity or at least one anomalous metric value of a second entity linked to the entity through a direct or an indirect link path.
20. One or more non-transitory computer-readable storage media having instructions stored thereon that, when executed by one or more processing devices, cause the processing devices to perform operations comprising:
obtaining, for each of a plurality of entities operating in a computer network being monitored for network security, a respective observed metric value for each of a plurality of metrics that each represent a respective behavior exhibited by the entity, wherein the plurality of entities comprise one or more users or one or more host computers;
for each of the plurality of metrics,
training using information about typical entity behavior in the monitored network from various sources, a respective machine learning model that is specific to the metric and that models an expected metric value for the metric;
for each of the plurality of entities and for each of the plurality of metrics:
determining, using the trained machine learning model that is specific to the metric and, an anomaly score for the observed metric value for the entity that represents how anomalous the observed metric value is relative to an expected metric value for the metric by:
determining a probability of the observed metric value occurring in the computer network being monitored using the trained machine learning model that is specific to the metric; and
determining the anomaly score from the determined probability of the observed metric value occurring in the computer network being monitored, the anomaly score indicating how anomalous the observed metric value for the entity is relative to an expected metric value for the specific metric;
aggregating the anomaly scores, from the machine learning models specific to the respective metrics to generate a respective threat score for each entity; and
detecting potential network threats based on the threat scores of the entities.
21. The non-transitory computer-readable medium of claim 20 , wherein the operations further comprise:
generating a security threat presentation that identifies one or more highest-scoring entities according to the threat scores; and
providing the security threat presentation for display on a user device.