Method and system for semi-supervised anomaly detection with feed-forward neural network for high-dimensional sensor data
One embodiment can provide a system for detecting anomaly for high-dimensional sensor data associated with one or more machines. During operation, the system can obtain sensor data from a set of sensor associated with one or machines, generate a first set of outputs by using a set of clustering models learned in parallel from the unlabeled sensor data and user-provided partial label information, generate a second set of outputs by using a set of feed-forward neural network (FNN) models learned in parallel from the first set of outputs and the unlabeled sensor data, and determine whether an anomaly is present in the operation of the one or more machines based on the second set of outputs and a user-specified threshold.
1. A method for detecting an anomaly in operation of one or more machines, the method comprising:
obtaining a plurality of unlabeled sensor data samples from one or more sensors associated with the one or more machines;
training a plurality of randomly initialized clustering models in parallel using the unlabeled sensor data and user-provided partial label information including a set of normal labels to generate a set of estimated labels, wherein
training each randomly initialized clustering model comprises evaluating a reliability of the randomly initialized clustering model, wherein
evaluating the reliability of each randomly initialized clustering model comprises computing a weight associated with the randomly initialized clustering model based on a matching rate between estimated labels outputted by the randomly initialized clustering model and user-provided ground-truth labels, and wherein
the estimated labels are determined by calculating
k
*
=
arg
max
k
π
k
N
(
x
*
❘
"\[LeftBracketingBar]"
μ
k
,
∑
k
)
,
with x * being the unlabeled sensor data;
computing, for each unlabeled sensor data sample, an abnormal probability based on the estimated labels and the weight associated with each randomly initialized clustering model;
applying a random sample generator to generate multiple sets of labeled training samples based on the abnormal probability;
training a set of feed-forward neural network (FNN) models in parallel, wherein a respective FNN model of the set of trained FNN models is trained using a corresponding set of labeled training samples;
obtaining, for an observed sensor data sample, a set of predicted labels outputted by the set of trained FNN models, wherein each trained FNN model outputs a predicted label;
computing an average of the set of predicted labels outputted by the set of trained FNN models; and
determining whether an anomaly is present in the operation of the one or more machines based on whether the average of the set of predicted labels is greater than a user-specified threshold.
2. The method of claim 1 , further comprising pre-processing the sensor data samples prior to generating the set of estimated labels.
3. The method of claim 1 , wherein each label in the set of estimated labels is:
a normal label; or
an abnormal label.
4. The method of claim 1 , wherein at least one randomly initialized clustering model in the plurality of randomly initialized clustering models includes a Gaussian Mixture Model (GMM).
5. The method of claim 1 , wherein computing the abnormal probability further comprises computing a weighted average of labels outputted by the plurality of randomly initialized cluttering models, and
wherein applying the random sample generator to generate the multiple sets of labeled training samples further comprises applying a Bernoulli random sample generator to generate the multiple sets of labeled training samples based on the abnormal probability.
6. An apparatus for detecting an anomaly in operation of one or more machines, comprising: one or more processors;
a set of sensors embedded in the one or more machines; and memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
obtain a plurality of unlabeled sensor data samples from one or more sensors associated with the one or more machines;
train a plurality of randomly initialized clustering models in parallel using the unlabeled sensor data samples and user-provided partial label information including a set of normal labels to generate a set of estimated labels, wherein
training each randomly initialized clustering model comprises evaluating a reliability of the randomly initialized clustering model, wherein
evaluating the reliability of each randomly initialized clustering model comprises computing a weight associated with the randomly initialized clustering model based on a matching rate between estimated labels outputted by the randomly initialized clustering model and user-provided ground-truth labels, and wherein
the estimated labels are determined by calculating
k
*
=
arg
max
k
π
k
N
(
x
*
❘
"\[LeftBracketingBar]"
μ
k
,
∑
k
)
,
with x * being the unlabeled sensor data;
computing, for each unlabeled sensor data sample, an abnormal probability based on the estimated labels and the weight associated with each randomly initialized clustering model;
apply a random sample generator to generate multiple sets of labeled training samples based on the abnormal probability;
train a set of feed-forward neural network (FNN) models in parallel, wherein a respective FNN model of the set of trained FNN models is trained using a corresponding set of labeled training samples;
obtain, for an observed sensor data sample, a set of predicted labels outputted by the set of trained FNN models, wherein each trained FNN model outputs a predicted label;
compute an average of the set of predicted labels outputted by the set of trained FNN models; and
determine whether an anomaly is present in the operation of the one or more machines based on whether the average of the set of predicted labels is greater than a user-specified threshold.
7. The apparatus of claim 6 , wherein obtaining the sensor data samples further comprises: pre-processing the sensor data samples.
8. The apparatus of claim 6 , wherein each label in the set of estimated labels is:
a normal label; or
an abnormal label.
9. The apparatus of claim 6 , wherein at least one randomly initialized clustering model in the plurality of randomly initialized clustering models includes a Gaussian Mixture Model (GMM).
10. The apparatus of claim 6 , wherein computing the abnormal probability further comprises computing a weighted average of labels outputted by the plurality of randomly initialized cluttering models, and wherein applying the random sample generator to generate the multiple sets of labeled training samples further comprises applying a Bernoulli random sample generator to generate the multiple sets of labeled training samples based on the abnormal probability.
11. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for detecting an anomaly in operation of one or more machines, the method comprising:
obtaining a plurality of unlabeled sensor data sample from one or more sensors associated with the one or more machines;
training a plurality of randomly initialized clustering models in parallel using the unlabeled sensor data and user-provided partial label information including a set of normal labels to generate a set of estimated labels, wherein
training each randomly initialized clustering model comprises evaluating a reliability of the randomly initialized clustering model, wherein
evaluating the reliability of each randomly initialized clustering model comprises computing a weight associated with the randomly initialized clustering model based on a matching rate between estimated labels outputted by the randomly initialized clustering model and user-provided ground-truth labels, and wherein
the estimated labels are determined by calculating
k
*
=
arg
max
k
π
k
N
(
x
*
❘
"\[LeftBracketingBar]"
μ
k
,
∑
k
)
,
with x * being the unlabeled sensor data;
computing, for each unlabeled sensor data sample, an abnormal probability based on the estimated labels and the weight associated with each randomly initialized clustering model;
applying a random sample generator to generate multiple sets of labeled training samples based on the plurality of unlabeled sensor data samples and the set of estimated labels;
training a set of feed-forward neural network (FNN) models in parallel, wherein a respective FNN model of the set of trained FNN models is trained using a corresponding set of labeled training samples;
obtaining, for an observed sensor data sample, a set of predicted labels outputted by the set of trained FNN models, wherein each trained FNN model outputs a predicted label;
computing an average of the set of predicted labels outputted by the set of trained FNN models; and
determining whether an anomaly is present in the operation of the one or more machines based on whether the average of the set of predicted labels is greater than a user-specified threshold.
12. The non-transitory computer-readable storage medium claim 11 , wherein at least one randomly initialized clustering model in the plurality of randomly initialized clustering models includes a Gaussian Mixture Model (GMM).
13. The non-transitory computer-readable storage medium claim 11 , wherein each label in the set of estimated labels is one of:
a normal label; and
an abnormal label.
14. The non-transitory computer-readable storage medium claim 11 , wherein computing the abnormal probability further comprises computing a weighted average of labels outputted by the plurality of randomly initialized cluttering models, and wherein applying the random sample generator to generate the multiple sets of labeled training samples further comprises applying a Bernoulli random sample generator to generate the multiple sets of labeled training samples based on the abnormal probability.