Density estimation network for unsupervised anomaly detection
Systems and methods for preventing cyberattacks using a Density Estimation Network (DEN) for unsupervised anomaly detection, including constructing the DEN using acquired network traffic data by performing end-to-end training. The training includes generating low-dimensional vector representations of the network traffic data by performing dimensionality reduction of the network traffic data, predicting mixture membership distribution parameters for each of the low-dimensional representations by performing density estimation using a Gaussian Mixture Model (GMM) framework, and formulating an objective function to estimate an energy and determine a density level of the low-dimensional representations for anomaly detection, with an anomaly being identified when the energy exceeds a pre-defined threshold. Cyberattacks are prevented by blocking transmission of network flows with identified anomalies by directly filtering out the flows using a network traffic monitor.
1. A method for preventing cyberattacks using a Density Estimation Network (DEN) for unsupervised anomaly detection, comprising:
constructing the DEN using acquired network traffic data by performing end-to-end training, the training comprising:
generating, using a compression network, low-dimensional vector representations of the network traffic data by performing dimensionality reduction of the network traffic data;
predicting mixture membership distribution parameters for each of the low-dimensional vector representations by performing density estimation using a Gaussian Mixture Model (GMM) framework, and
formulating an objective function J to estimate an energy and determine a density level of the low-dimensional representations for anomaly detection, an anomaly being identified when the energy exceeds a pre-defined threshold, wherein the objective function J is determined as follows:
J =reconstruction_error+energy,
where reconstruction error can represent any reconstruction error made by dimensionality reduction, and energy is also called negative likelihood, which is inversely proportional to the likelihood that one can observe given training data under the current parameterized Gaussian mixture model; and
preventing cyberattacks by blocking transmission of network flows with identified anomalies by directly filtering out the flows using a network traffic monitor.
2. The method as recited in claim 1 , wherein the low-dimensional vector representations are generated from reduced space and reconstruction error features.
3. The method as recited in claim 1 , wherein the dimensionality reduction of the network traffic data is performed using a deep autoencoder.
4. The method as recited in claim 1 , further comprising identifying the mixture means and mixture deviations using the predicted mixture membership distribution parameters.
5. The method as recited in claim 1 , wherein the energy is determined as follows:
E
(
z
)
=
-
log
(
∑
k
=
1
K
ϕ
^
k
exp
(
-
1
2
(
z
-
μ
^
k
)
T
Σ
^
k
-
1
(
z
-
μ
^
k
)
)
2
π
Σ
^
k
)
where z is the hidden representation of an input sample, Pk and 2 k are the mean and covariance matrix in the k-th component of the GMM, 71 is approximately 3.14159, Pk is the probability that a sample belongs to the k-th component, and K is the number of components in the GMM.
6. The method as recited in claim 1 , wherein the DEN is a multi-layer neural network.
7. The method as recited in claim 1 , wherein the network traffic data is high-dimensional, contaminated data.
8. A system for preventing cyberattacks using a Density Estimation Network (DEN) for unsupervised anomaly detection, comprising:
a hardware processor operatively coupled to a computer readable storage medium, the hardware processor being configured for constructing, using a DEN constructor, the DEN based on acquired network traffic data by performing end-to-end training using a DEN trainer, the training comprising:
generating, using a compression network, low-dimensional vector representations of the network traffic data by performing dimensionality reduction of the network traffic data;
predicting, using a density estimator, mixture membership distribution parameters for each of the low-dimensional vector representations by performing density estimation using a Gaussian Mixture Model (GMM) framework; and
formulating, using an objective function calculator, an objective function J to estimate an energy and determine a density level of the low-dimensional representations for anomaly detection, an anomaly being identified when the energy exceeds a pre-defined threshold, wherein the objective function J is determined as follows:
J =reconstruction error+energy,
where reconstruction error can represent any reconstruction error made by dimensionality reduction, and energy is also called negative likelihood, which is inversely proportional to the likelihood that one can observe given training data under the current parameterized Gaussian mixture model; and
a controller configured for preventing cyberattacks by blocking transmission of network flows with identified anomalies by directly filtering out the flows using a network traffic monitor.
9. The system as recited in claim 8 , wherein the low-dimensional vector representations are generated from reduced space and reconstruction error features.
10. The system as recited in claim 8 , wherein the dimensionality reduction of the network traffic data is performed using a deep autoencoder.
11. The system as recited in claim 8 , further comprising identifying, using a parameter learner, the mixture means and mixture deviations using the predicted mixture membership distribution parameters.
12. The system as recited in claim 8 , wherein the energy is determined as follows:
E
(
z
)
=
-
log
(
∑
k
=
1
K
ϕ
^
k
exp
(
-
1
2
(
z
-
μ
^
k
)
T
Σ
^
k
-
1
(
z
-
μ
^
k
)
)
2
π
Σ
^
k
)
where z is the hidden representation of an input sample, Pk and 2 k are the mean and covariance matrix in the k-th component of the GMM, 7 r is approximately 3.14159, 1k is the probability that a sample belongs to the k-th component, and K is the number of components in the GMM.
13. The system as recited in claim 8 , wherein the DEN is a multi-layer neural network.
14. The system as recited in claim 8 , wherein the network traffic data is high-dimensional, contaminated data.
15. A non-transitory computer readable storage medium comprising a computer readable program for preventing cyberattacks using a Density Estimation Network (DEN) for unsupervised anomaly detection, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:
constructing the DEN using acquired network traffic data by performing end-to-end training, the training comprising:
generating, using a compression network, low-dimensional vector representations of the network traffic data by performing dimensionality reduction of the network traffic data;
predicting mixture membership distribution parameters for each of the low-dimensional vector representations by performing density estimation using a Gaussian Mixture Model (GMM) framework; and
formulating an objective function J to estimate an energy and determine a density level of the low-dimensional representations for anomaly detection, an anomaly being identified when the energy exceeds a pre-defined threshold, wherein the objective function J is determined as follows:
J =reconstruction error+energy
where reconstruction error can represent any reconstruction error made by dimensionality reduction, and energy is also called negative likelihood, which is inversely proportional to the likelihood that one can observe given training data under the current parameterized Gaussian mixture model; and
preventing the cyberattacks by blocking transmission of network flows with identified anomalies by directly filtering out the flows using a network traffic monitor.
16. The computer readable storage medium as recited in claim 15 , wherein the low-dimensional vector representations are generated from reduced space and reconstruction error features.
17. The computer readable storage medium as recited in claim 15 , wherein the dimensionality reduction of the network traffic data is performed using a deep autoencoder.
18. The computer readable storage medium as recited in claim 15 , wherein the energy is determined as follows:
E
(
z
)
=
-
log
(
∑
k
=
1
K
ϕ
^
k
exp
(
-
1
2
(
z
-
μ
^
k
)
T
Σ
^
k
-
1
(
z
-
μ
^
k
)
)
2
π
Σ
^
k
)
where z is the hidden representation of an input sample, k and 2 k are the mean and covariance matrix in the k-th component of the GMM, 71 is approximately 3.14159, k is the probability that a sample belongs to the k-th component, and K is the number of components in the GMM.