Anomaly detection in computer networks
View Patent ↗A method of anomaly detection for network traffic communicated by devices via a computer network, the method including clustering a set of time series, each time series including a plurality of time windows of data corresponding to network communication characteristics for a device; training an autoencoder for each cluster based on time series in the cluster; generating a set of reconstruction errors for each autoencoder based on testing the autoencoder with data from time windows of at least a subset of the time series; generating a probabilistic model of reconstruction errors for each autoencoder; and generating an aggregation of the probabilistic models for, in use, detecting reconstruction errors for a time series of data corresponding to network communication characteristics for a device as anomalous.
1. A method of anomaly detection for network traffic communicated by devices via a computer network, the method comprising:
clustering a set of time series, each time series including a plurality of time windows of data corresponding to network communication characteristics for a device and each cluster being defined based on an autoencoder for each cluster converting each time series to a vector of features for the time series and a clustering algorithm clustering the vectors;
training the autoencoder for each cluster based on a time series in the cluster;
generating, for the autoencoder of each cluster, a set of reconstruction errors for the autoencoder based on testing the autoencoder with data from time windows of at least a subset of the time series from which the autoencoder was trained;
generating a probabilistic model of reconstruction errors for each autoencoder; and
generating an aggregation of the probabilistic model for, in use, detecting reconstruction errors for a time series of data corresponding to network communication characteristics for a device as anomalous.
2. The method of claim 1 , wherein the set of reconstruction errors for the autoencoder of each cluster is generated based on the autoencoder processing each time series in a corresponding cluster of time series.
3. The method of claim 1 , wherein each cluster is defined based on a random subdivision of the set of time series.
4. The method of claim 3 , wherein the set of reconstruction errors for the autoencoder of each cluster is generated based on the autoencoder processing each of the time series.
5. The method of claim 1 , wherein each probabilistic model is a Gaussian model of reconstruction errors for the autoencoder of each cluster.
6. The method of claim 5 , wherein the aggregation of the probabilistic model is a Gaussian mixture model.
7. The method of claim 1 , wherein the aggregation of the probabilistic model is a hidden Markov model.
8. A computer system comprising:
a processor and memory storing computer program code for anomaly detection for network traffic communicated by devices via a computer network, by:
clustering a set of time series, each time series including a plurality of time windows of data corresponding to network communication characteristics for a device and each cluster being defined based on an autoencoder for each cluster converting each time series to a vector of features for the time series and a clustering algorithm clustering the vectors;
training an autoencoder for each cluster based on a time series in the cluster;
generating, for the autoencoder of each cluster, a set of reconstruction errors for each autoencoder based on testing a respective autoencoder with data from time windows of at least a subset of the time series from which each respective autoencoder was trained;
generating a probabilistic model of reconstruction errors for the autoencoder; and
generating an aggregation of the probabilistic model for, in use, detecting reconstruction errors for a time series of data corresponding to network communication characteristics for a device as anomalous.
9. A non-transitory computer-readable storage medium storing a computer program element comprising computer program code to, when loaded into a computer system and executed thereon, cause the computer system to perform anomaly detection for network traffic communicated by devices via a computer network, by:
clustering a set of time series, each time series including a plurality of time windows of data corresponding to network communication characteristics for a device and each cluster being defined based on an autoencoder for each cluster converting each time series to a vector of features for the time series and a clustering algorithm clustering the vectors;
training an autoencoder for each cluster based on a time series in the cluster;
generating, for the autoencoder of each cluster, a set of reconstruction errors for the autoencoder based on testing the autoencoder with data from time windows of at least a subset of the time series from which the autoencoder was trained;
generating a probabilistic model of reconstruction errors for each autoencoder; and
generating an aggregation of the probabilistic model for, in use, detecting reconstruction errors for a time series of data corresponding to network communication characteristics for a device as anomalous.