Systems and methods for anomaly detection
Computer network anomaly detection systems and methods are disclosed. One embodiment includes retrieving one or more learned profiles for a group of networked computing devices included in a computer network from a database. For each pair of computing devices in the group, a pairwise distance matrix may be computed. Each pairwise distance in the pairwise distance matrix is computed based on a statistical data profile associated with each computing device in each pair of computing devices from the group. The statistical data profiles may be included in the learned profiles. Any pairwise distances that are greater than a threshold may be removed from the pairwise distance matrix to generate a reduced pairwise distance matrix. One or more computing devices associated with the remaining pairwise distances in the reduced pairwise distance matrix may be sorted into a cluster of computing devices. An anomaly score may be computed for the cluster.
1 . A method comprising:
generating one or more learned profiles for a group of networked computing devices included in a computer network, the generating comprising:
collecting computer network data communicated by the computing devices over the computer network for a predetermined time period;
categorizing the computing devices into the group;
calculating individual statistical data for each computing device including an average and a standard deviation;
computing a weighted average and standard deviation for statistical data associated with the group;
using the weighted average and standard deviation for the group as a reference point, computing a distance of each average and standard deviation associated with each computing device from the reference point; and
storing the learned profiles in a database;
retrieving, from the database, the one or more learned profiles;
computing, for each pair of computing devices in the group, a pairwise distance matrix, wherein each pairwise distance in the pairwise distance matrix is computed based on a statistical data profile associated with each computing device in each pair of computing devices from the group, and wherein the statistical data profiles are included in the learned profiles;
removing any pairwise distances that are greater than a threshold from the pairwise distance matrix to generate a reduced pairwise distance matrix;
clustering the computing devices associated with the remaining pairwise distances in the reduced pairwise distance matrix into a cluster of computing devices; and
computing an anomaly score for the cluster.
2 . The method of claim 1 , wherein the clustering is associated with a predetermined time period.
3 . The method of claim 2 , wherein the time period is any of a plurality of weeks, a plurality of days, a plurality of hours, or a plurality of minutes.
4 . The method of claim 1 , wherein the clustering is based on computing device properties that include the learned profiles, a number of connections, a number of bytes sent or received, a number of destination IP addresses, and a number of destination ports during the time period.
5 . The method of claim 1 , wherein the threshold is determined and the clustering is performed based on one or more computing device characteristics and behavior attributes.
6 . The method of claim 1 , further comprising:
categorizing the computing devices into a plurality of clusters; and
computing an individual anomaly score for each cluster.
7 . The method of claim 1 , wherein the anomaly score is computed based on determining a number of computing system nodes in the cluster.
8 . The method of claim 1 , wherein each element in the pairwise distance matrix is any of a Euclidian distance, a Mahalanobis distance, or a point-to-point distance.
9 . The method of claim 1 , wherein the pairwise distance matrix is a symmetric matrix.
10 . The method of claim 1 , wherein membership of a computing device in the group is based on any combination of a common organization with other computing devices in the group, or a similar function to other computing devices in the group.
11 . The method of claim 10 , wherein is a group is any of an information technology (IT) group, a human resources (HR) group, or an operations group.