Sparse neural network based anomaly detection in multi-dimensional time series
Anomaly detection from time series is one of the key components in automated monitoring of one or more entities. Domain-driven sensor selection for anomaly detection is restricted by knowledge of important sensors to capture only a certain set of anomalies from the entire set of possible anomalies. Hence, existing anomaly detection approaches are not very effective for multi-dimensional time series. Embodiments of the present disclosure depict sparse neural network for anomaly detection in multi-dimensional time series (MDTS) corresponding to a plurality of parameters of entities. A reduced-dimensional time series is obtained from the MDTS via an at least one feedforward layer by using a dimensionality reduction model. The dimensionality reduction model and recurrent neural network (RNN) encoder-decoder model are simultaneously learned to obtain a multi-layered sparse neural network. A plurality of error vectors corresponding to at least one time instance of the MDTS is computed to obtain an anomaly score.
1. A processor-implemented method, comprising:
receiving, at an input layer, a multi-dimensional time series corresponding to a plurality of parameters of an entity;
obtaining, using a dimensionality reduction model, a reduced-dimensional time series from the multi-dimensional time series via an at least one feedforward layer, wherein connections between the input layer and the feedforward layer are sparse to access at least a portion of the plurality of parameters;
estimating, by using a recurrent neural network (RNN) encoder-decoder model, the multi-dimensional time series using the reduced-dimensional time series obtained by the dimensionality reduction model;
simultaneously learning, by using the estimated multi-dimensional time series, the dimensionality reduction model and the RNN encoder-decoder model to obtain a multi-layered sparse neural network, wherein the learning comprises training a sparse feedforward layer of the dimensionality reduction model and the RNN encoder-decoder model using a stochastic gradient descent, wherein a value of gradient of the sparse feedforward layer of the dimensionality reduction model is based on an element of a sparse weight matrix, wherein the sparse feedforward layer acts as a regularizer such that the reduced-dimensional time series in reduced dimensional space in a Rectified Linear Unit (ReLU) layer is forced to capture information to reconstruct the multi-dimensional time series, and wherein the RNN encoder-decoder model is trained as a temporal autoencoder on a normal time series and provides a reconstruction error and a corresponding anomaly score for an anomalous time series pattern;
computing, by using the multi-layered sparse neural network, a plurality of error vectors corresponding to at least one time instance of the multi-dimensional time series by performing a comparison of the multi-dimensional time series and the estimated multi-dimensional time series; and
generating at least one anomaly score based on the plurality of the error vectors to classify the at least one time instance in the multi-dimensional time series as anomalous or normal using a threshold learned based on a hold-out validation set while maximizing F-score, wherein the hold-out validation set comprises at least one normal time instance and at least one anomalous time instance of the multi-dimensional time series.
2. The processor-implemented method of claim 1 , wherein each of a plurality of parameters in the reduced-dimensional time series is a non-linear function of a subset of a plurality of parameters of the multi-dimensional time series.
3. The processor-implemented method of claim 1 , wherein the dimensionality reduction model comprises a plurality of feedforward layers with Least Absolute Shrinkage and Selection Operator (LASSO) sparsity constraint on a plurality of parameters of the plurality of feedforward layers.
4. The processor-implemented method of claim 1 , further comprising:
(a) classifying the at least one time instance in the multi-dimensional time series as anomalous if the anomaly score is greater than the threshold, or
(b) classifying the at least one time instance in the multi-dimensional time series as normal if the anomaly score is less than or equal to the threshold.
5. A system comprising:
a memory storing instructions; one or more communication interfaces; and
one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
receive, at an input layer, a multi-dimensional time series corresponding to a plurality of parameters of an entity;
obtain, using a dimensionality reduction model, a reduced-dimensional time series from the multi-dimensional time series via an at least one feedforward layer, wherein connections between the input layer and the feedforward layer are sparse to access at least a portion of the plurality of parameters;
estimate, by using a recurrent neural network (RNN) encoder-decoder model, the multi-dimensional time series using the reduced-dimensional time series obtained by the dimensionality reduction model;
simultaneously learn, by using the estimated multi-dimensional time series, the dimensionality reduction model and the RNN encoder-decoder model to obtain a multi-layered sparse neural network, wherein the learning comprises training a sparse feedforward layer of the dimensionality reduction model and the RNN encoder-decoder model using a stochastic gradient descent, wherein a value of gradient of the sparse feedforward layer of the dimensionality reduction model is based on an element of a sparse weight matrix, wherein the sparse feedforward layer acts as a regularizer such that the reduced-dimensional time series in reduced dimensional space in a Rectified Linear Unit (ReLU) layer is forced to capture information to reconstruct the multi-dimensional time series, and wherein the RNN encoder-decoder model is trained as a temporal autoencoder on a normal time series and provides a reconstruction error and a corresponding anomaly score for an anomalous time series pattern;
compute, by using the multi-layered sparse neural network, a plurality of error vectors corresponding to at least one time instance of the multi-dimensional time series by performing a comparison of the multi-dimensional time series and the estimated multi-dimensional time series; and
generate at least one anomaly score based on the plurality of the error vectors to classify the at least one time instance in the multi-dimensional time series as anomalous or normal using a threshold learned based on a hold-out validation set while maximizing F-score, wherein the hold-out validation set comprises at least one normal time instance and at least one anomalous time instance of the multi-dimensional time series.
6. The system of claim 5 , wherein each of a plurality of parameters in the reduced-dimensional time series is a non-linear function of a subset of a plurality of parameters of the multi-dimensional time series.
7. The system of claim 5 , wherein the dimensionality reduction model comprises a plurality of feedforward layers with Least Absolute Shrinkage and Selection Operator (LASSO) sparsity constraint on a plurality of parameters of the plurality of feedforward layers.
8. The system of claim 5 , wherein the one or more hardware processors are further configured to:
(a) classify the at least one time instance in the multi-dimensional time series as anomalous if the anomaly score is greater than the threshold, or
(b) classify the at least one time instance in the multi-dimensional time series as normal if the anomaly score is less than or equal to the threshold.
9. One or more non-transitory machine readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
receiving, at an input layer, a multi-dimensional time series corresponding to a plurality of parameters of an entity;
obtaining, using a dimensionality reduction model, a reduced-dimensional time series from the multi-dimensional time series via an at least one feedforward layer, wherein connections between the input layer and the feedforward layer are sparse to access at least a portion of the plurality of parameters;
estimating, by using a recurrent neural network (RNN) encoder-decoder model, the multi-dimensional time series using the reduced-dimensional time series obtained by the dimensionality reduction model;
simultaneously learning, by using the estimated multi-dimensional time series, the dimensionality reduction model and the RNN encoder-decoder model to obtain a multi-layered sparse neural network, wherein the learning comprises training a sparse feedforward layer of the dimensionality reduction model and the RNN encoder-decoder model using a stochastic gradient descent, wherein a value of gradient of the sparse feedforward layer of the dimensionality reduction model is based on an element of a sparse weight matrix, wherein the sparse feedforward layer acts as a regularizer such that the reduced-dimensional time series in reduced dimensional space in a Rectified Linear Unit (ReLU) layer is forced to capture information to reconstruct the multi-dimensional time series, and wherein the RNN encoder-decoder model is trained as a temporal autoencoder on a normal time series and provides a reconstruction error and a corresponding anomaly score for an anomalous time series pattern;
computing, by using the multi-layered sparse neural network, a plurality of error vectors corresponding to at least one time instance of the multi-dimensional time series by performing a comparison of the multi-dimensional time series and the estimated multi-dimensional time series; and
generating at least one anomaly score based on the plurality of the error vectors to classify the at least one time instance in the multi-dimensional time series as anomalous or normal using a threshold learned based on a hold-out validation set while maximizing F-score, wherein the hold-out validation set comprises at least one normal time instance and at least one anomalous time instance of the multi-dimensional time series.
10. The one or more non-transitory machine readable information storage mediums of claim 9 , wherein each of a plurality of parameters in the reduced-dimensional time series is a non-linear function of a subset of a plurality of parameters of the multi-dimensional time series.
11. The one or more non-transitory machine readable information storage mediums of claim 9 , wherein the dimensionality reduction model comprises a plurality of feedforward layers with Least Absolute Shrinkage and Selection Operator (LASSO) sparsity constraint on a plurality of parameters of the plurality of feedforward layers.
12. The one or more non-transitory machine readable information storage mediums of claim 9 , further comprising:
(a) classifying the at least one time instance in the multi-dimensional time series as anomalous if the anomaly score is greater than the threshold, or
(b) classifying the at least one time instance in the multi-dimensional time series as normal if the anomaly score is less than or equal to the threshold.