Anomaly detection in computer networks
View Patent ↗A method of anomaly detection for network traffic communicated by devices via a computer network, the method including receiving a set of training time series each including a plurality of time windows of data corresponding to network communication characteristics for a first device; training an autoencoder for a first cluster based on a time series in the first cluster, wherein a state of the autoencoder is periodically recorded after a predetermined fixed number of training examples to define a set of trained autoencoders for the first cluster; receiving a new time series including a plurality of time windows of data corresponding to network communication characteristics for the first device; for each time window of the new time series, generating a vector of reconstruction errors for the first device for each autoencoder based on testing the autoencoder with data from the time window; and evaluating a derivative of each vector; training a machine learning model based on the derivatives so as to define a filter for identifying subsequent time series for a second device being absent anomalous communication.
1. A method of anomaly detection for network traffic communicated by devices via a computer network, the method comprising:
receiving a set of training time series each including a plurality of time windows of data corresponding to network communication characteristics for a first device;
training an autoencoder for a first cluster based on a time series in the first cluster, wherein a state of the autoencoder is periodically recorded after a predetermined fixed number of training examples to define a set of trained autoencoders for the first cluster;
receiving a new time series including a plurality of time windows of data corresponding to network communication characteristics for the first device; for each time window of the new time series, generating a vector of reconstruction errors for the first device for each autoencoder based on testing the autoencoder with data from the time window;
evaluating a derivative of each vector; and
training a machine learning model based on the derivatives so as to define a predetermined threshold for identifying subsequent time series for a second device, wherein the subsequent time series identified for the second device are absent anomalous communication in response to determining that a comparison between the vector of reconstruction errors and reconstruction errors detected during the subsequent time series does not exceed the predetermined threshold.
2. A computer system comprising:
a processor and memory storing computer program code for detecting anomalies in network traffic communicated by devices via a computer network, by:
receiving a set of training time series each including a plurality of time windows of data corresponding to network communication characteristics for a first device;
training an autoencoder for a first cluster based on a time series in the first cluster, wherein a state of the autoencoder is periodically recorded after a predetermined fixed number of training examples to define a set of trained autoencoders for the first cluster;
receiving a new time series including a plurality of time windows of data corresponding to network communication characteristics for the first device;
for each time window of the new time series, generating a vector of reconstruction errors for the first device for each autoencoder based on testing the autoencoder with data from the time window;
evaluating a derivative of each vector; and
training a machine learning model based on the derivatives so as to define a predetermined threshold for identifying subsequent time series for a second device, wherein the subsequent time series identified for the second device are absent anomalous communication in response to determining that a comparison between the vector of reconstruction errors and reconstruction errors detected during the subsequent time series does not exceed the predetermined threshold.
3. A non-transitory computer-readable storage element storing computer program code to, when loaded into a computer system and executed thereon, cause the computer system to detect anomalies in network traffic communicated by devices via a computer network, by:
receiving a set of training time series each including a plurality of time windows of data corresponding to network communication characteristics for a first device;
training an autoencoder for a first cluster based on a time series in the first cluster, wherein a state of the autoencoder is periodically recorded after a predetermined fixed number of training examples to define a set of trained autoencoders for the first cluster;
receiving a new time series including a plurality of time windows of data corresponding to network communication characteristics for the first device;
for each time window of the new time series, generating a vector of reconstruction errors for the first device for each autoencoder based on testing the autoencoder with data from the time window;
evaluating a derivative of each vector; and
training a machine learning model based on the derivatives so as to define a predetermined threshold for identifying subsequent time series for a second device, wherein the subsequent time series identified for the second device are absent anomalous communication in response to determining that a comparison between the vector of reconstruction errors and reconstruction errors detected during the subsequent time series does not exceed the predetermined threshold.