Unsupervised anomaly detection of industrial dynamic systems with contrastive latent density learning
Anomaly detection in industrial dynamic process can include receiving a set of multivariate time series data representative of sensor data obtained over time. The set of multivariate time series data can be transformed into a set of signature vectors in an embedding space. A neural network can be trained to estimate a probability distribution of the set of signature vectors in the embedding space. Streaming data can be received. The streaming data can be appended with a previously stored time series data. The appended streaming data can be transformed into an embedding. The embedding can be input into the trained neural network, the trained neural network outputting a first probability distribution score. A second probability distribution score associated with the embedding can be determined based on a given proposed probability distribution. Anomaly score can be determined based on the first probability distribution score and the second probability distribution score.
1 . A method of detecting anomaly in an industrial process, comprising:
receiving a set of multivariate time series data representative of sensor data obtained over time;
transforming the set of multivariate time series data into a set of signature vectors in an embedding space, the set of multivariate time series data being an unlabeled data set;
training a neural network to estimate a probability distribution of the set of signature vectors in the embedding space, the training using the set of signature vectors transformed from the unlabeled data set,
wherein the neural network is trained by contrasting random samples from the set of signature vectors with random samples from a given distribution and updating weights of the neural network based on the contrasting, wherein input for training of the neural network includes at least the set of multivariate time series data and the given distribution, the set of multivariate time series data including the sensor data and corresponding control variable data, the given distribution being configurable, wherein the contrasting random samples from the set of signature vectors with random samples from a given distribution includes:
sampling a signature vector randomly and passing the randomly sampled signature vector to the neural network;
computing a loss function of the randomly sampled signature vector passed to the neural network;
sampling a random sample from the given distribution and passing the random sample from the given distribution to the neural network; and
computing a loss function of the random sample from the given distribution passed to the neural network;
wherein the updating the weights of the neural network based on the contrasting includes updating the weights of the neural network using a total loss, the total loss being a sum of the loss function of the randomly sampled signature vector passed to the neural network and the loss function of the random sample from the given distribution passed to the neural network.
2 . The method of claim 1 , wherein the set of multivariate time series data is transformed into the set of signature vectors by learning an embedding function using deep learning.
3 . The method of claim 1 , further including receiving streaming data and based on the trained neural network, determining an anomaly score in the streaming data.
4 . The method of claim 1 , further including:
receiving streaming data;
appending the streaming data with a previously stored time series data;
transforming the appended streaming data into an embedding;
inputting the embedding into the trained neural network, the trained neural network outputting a first probability distribution score;
determining a second probability distribution score associated with the embedding based on a given proposed probability distribution;
determining an anomaly score based on the first probability distribution score and the second probability distribution score.
5 . The method of claim 4 , further including:
comparing the anomaly score with a given threshold value; and
based on the comparison of the anomaly score with the given threshold value, determining anomalousness of the streaming data.
6 . A system comprising:
a processor; and
a memory device coupled with the processor;
the processor configured to at least:
receive a set of multivariate time series data representative of sensor data obtained over time;
transform the set of multivariate time series data into a set of signature vectors in an embedding space, the set of multivariate time series data being an unlabeled data set;
train a neural network to estimate a probability distribution of the set of signature vectors in the embedding space, the training using the set of signature vectors transformed from the unlabeled data set,
wherein the neural network is trained by contrasting random samples from the set of signature vectors with random samples from a given distribution and updating weights of the neural network based on the contrasting, wherein input for training of the neural network includes at least the set of multivariate time series data and the given distribution, the set of multivariate time series data including the sensor data and corresponding control variable data, the given distribution being configurable, wherein the contrasting random samples from the set of signature vectors with random samples from a given distribution includes:
sampling a signature vector randomly and passing the randomly sampled signature vector to the neural network;
computing a loss function of the randomly sampled signature vector passed to the neural network;
sampling a random sample from the given distribution and passing the random sample from the given distribution to the neural network; and
computing a loss function of the random sample from the given distribution passed to the neural network;
wherein the updating the weights of the neural network based on the contrasting includes updating the weights of the neural network using a total loss, the total loss being a sum of the loss function of the randomly sampled signature vector passed to the neural network and the loss function of the random sample from the given distribution passed to the neural network.
7 . The system of claim 6 , wherein the processor is configured to transform the set of multivariate time series data into the set of signature vectors by learning an embedding function using deep learning.
8 . The system of claim 7 , wherein the deep learning includes a recurrent neural network.
9 . The system of claim 7 , wherein the deep learning includes an autoencoder.
10 . The system of claim 6 , wherein the processor is further configured to receive a streaming data and based on the trained neural network, determine an anomaly score in the streaming data.
11 . The system of claim 6 , wherein the processor is further configured to:
receive streaming data;
append the streaming data with a previously stored time series data;
transform the appended streaming data into an embedding;
input the embedding into the trained neural network, the trained neural network outputting a first probability distribution score;
determine a second probability distribution score associated with the embedding based on a given proposed probability distribution;
determine an anomaly score based on the first probability distribution score and the second probability distribution score.
12 . The system of claim 11 , wherein the processor is further configured to:
compare the anomaly score with a given threshold value; and
based on the comparison of the anomaly score with the given threshold value, determine anomalousness of the streaming data.
13 . A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions readable by a device to cause the device to:
receive a set of multivariate time series data representative of sensor data obtained over time;
transform the set of multivariate time series data into a set of signature vectors in an embedding space, the set of multivariate time series data being an unlabeled data set;
train a neural network to estimate a probability distribution of the set of signature vectors in the embedding space, training the neural network using the set of signature vectors transformed from the unlabeled data set,
wherein the neural network is trained by contrasting random samples from the set of signature vectors with random samples from a given distribution and updating weights of the neural network based on the contrasting, wherein input for training of the neural network includes at least the set of multivariate time series data and the given distribution, the set of multivariate time series data including the sensor data and corresponding control variable data, the given distribution being configurable, wherein the contrasting random samples from the set of signature vectors with random samples from a given distribution includes:
sampling a signature vector randomly and passing the randomly sampled signature vector to the neural network;
computing a loss function of the randomly sampled signature vector passed to the neural network;
sampling a random sample from the given distribution and passing the random sample from the given distribution to the neural network; and
computing a loss function of the random sample from the given distribution passed to the neural network;
wherein the updating the weights of the neural network based on the contrasting includes updating the weights of the neural network using a total loss, the total loss being a sum of the loss function of the randomly sampled signature vector passed to the neural network and the loss function of the random sample from the given distribution passed to the neural network.
14 . The computer program product of claim 13 , wherein the device is caused to transform the set of multivariate time series data into the set of signature vectors by learning an embedding function using deep learning.
15 . The computer program product of claim 13 , wherein the device is further caused to receive streaming data and based on the trained neural network, determine an anomaly score in the streaming data.
16 . The computer program product of claim 13 , wherein the device is further caused to:
receive streaming data;
append the streaming data with a previously stored time series data;
transform the appended streaming data into an embedding;
input the embedding into the trained neural network, the trained neural network outputting a first probability distribution score;
determine a second probability distribution score associated with the embedding based on a given proposed probability distribution;
determine an anomaly score based on the first probability distribution score and the second probability distribution score.
17 . The computer program product of claim 16 , wherein device is further caused to:
compare the anomaly score with a given threshold value; and
based on the comparison of the anomaly score with the given threshold value, determine anomalousness of the streaming data.