System and method for detection of abnormal device traffic behavior
A system and method for detecting abnormal device traffic behavior. The method includes creating a baseline clustering model for a device based on a training data set including traffic data for the device, wherein the baseline clustering model includes a plurality of clusters, each cluster representing a discrete state and including a plurality of first data points of the training data set; sampling a plurality of second data points with respect to windows of time in order to create at least one sample, each sample including at least a portion of the plurality of second data points, wherein the plurality of second data points are related to traffic involving the device; and detecting anomalous traffic behavior of the device based on the at least one sample and the baseline clustering model.
1. A method, comprising:
creating a baseline clustering model specific to a device based on a training data set including traffic data for the device, wherein the baseline clustering model includes a plurality of clusters, each cluster of the plurality of clusters representing a discrete state and including a plurality of first data points of the training data set;
sampling a plurality of second data points with respect to windows of time thereby creating at least one sample, each sample of the at least one sample including at least a portion of the plurality of second data points, wherein the plurality of second data points are related to traffic involving the device;
calculating, for each second data point of the plurality of second data points in the at least one sample, a risk factor score based on a vector representation of the respective second data point in a traffic feature space and a proximity of the respective second data point to one of the plurality of clusters in the baseline clustering model specific to the device;
determining, for each second data point in the at least one sample, whether the respective second data point is an outlier based on the plurality of clusters in the baseline clustering model specific to the device;
detecting anomalous traffic behavior of the device by identifying a second data point in the at least one sample as an anomalous data point based on the risk factor score of that second data point and that second data point being a determined outlier;
updating the plurality of clusters of the baseline clustering model based on the at least one sample and the risk factor score for each second data point of each sample, wherein the updating comprises adding each sample having a high risk factor score as a new cluster to the plurality of clusters; and
performing at least one mitigation action based on the detected anomalous traffic behavior.
2. The method of claim 1 , wherein the detecting the anomalous traffic behavior further comprises:
for each second data point of each sample, determining whetherthe second data point is an outlier with respect to one of the plurality of clusters, wherein the anomalous traffic behavior is detected further based on at least one outlier second data point of the plurality of second data points.
3. The method of claim 1 , wherein the risk factor score for each second data point in the plurality of second data points is determined based on a vector representation of the second data point in a traffic feature space and a proximity of the second data point to one of the plurality of clusters.
4. The method of claim 3 , wherein the risk factor score is determined further based on a norm of the vector representation of the second data point, a length of a projection of the vector representation over a centroid of closest cluster of the plurality of clusters, and a length of the projection over a diagonal vector of the traffic feature space.
5. The method of claim 1 , wherein the updating the plurality of clusters further comprises:
integrating each sample having a risk factor score below a threshold into one of the plurality of clusters.
6. The method of claim 1 , wherein the detecting the anomalous traffic behavior further comprises:
comparing the risk factor score of each second data point of each sample to a threshold thereby determining whether the respective second data point for the risk factor score is high risk, wherein the respective second data point is high risk when the risk factor score for the respective second data point is above the threshold.
7. The method of claim 6 , further comprising:
updating the threshold based on the detected anomalous traffic behavior.
8. The method of claim 6 , wherein the threshold is a proportional value of a highest risk factor score among each second data point of each sample.
9. A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to execute a process, the process comprising:
creating a baseline clustering model specific to a device based on a training data set including traffic data for the device, wherein the baseline clustering model includes a plurality of clusters, each cluster of the plurality of clusters representing a discrete state and including a plurality of first data points of the training data set;
sampling a plurality of second data points with respect to windows of time thereby creating at least one sample, each sample of the at least one sample including at least a portion of the plurality of second data points, wherein the plurality of second data points are related to traffic involving the device;
calculating, for each second data point of the plurality of second data points in the at least one sample, a risk factor score based on a vector representation of the respective second data point in a traffic feature space and a proximity of the respective second data point to one of the plurality of clusters in the baseline clustering model specific to the device;
determining, for each second data point in the at least one sample, whether the respective second data point is an outlier based on the plurality of clusters in the baseline clustering model specific to the device;
detecting anomalous traffic behavior of the device by identifying a second data point in the at least one sample as an anomalous data point based on the risk factor score of that second data point and that second data point being a determined outlier;
updating the plurality of clusters of the baseline clustering model based on the at least one sample and risk factor score for each second data point of each sample, wherein the updating includes adding each sample having a high risk factor score as a new cluster to the plurality of clusters; and
performing at least one mitigation action based on the detected anomalous traffic behavior.
10. The non-transitory computer readable medium of claim 9 , wherein the process further comprises:
for each second data point of each sample, determine whetherthe second data point is an outlier with respect to one of the plurality of clusters, wherein the anomalous traffic behavior is detected further based on at least one outlier second data point of the plurality of second data points.
11. A system for detecting abnormal device traffic behavior, comprising:
a processing circuitry; and
a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:
create a baseline clustering model specific to a device based on a training data set including traffic data for the device, wherein the baseline clustering model includes a plurality of clusters, each cluster of the plurality of clusters representing a discrete state and including a plurality of first data points of the training data set;
sample a plurality of second data points with respect to windows of time thereby creating at least one sample, each sample of the at least one sample including at least a portion of the plurality of second data points, wherein the plurality of second data points are related to traffic involving the device;
calculate, for each second data point of the plurality of second data points in the at least one sample, a risk factor score based on a vector representation of the respective second data point in a traffic feature space and a proximity of the respective second data point to one of the plurality of clusters in the baseline clustering model specific to the device;
determine, for each second data point in the at least one sample, whether the respective second data point is an outlier based on the plurality of clusters in the baseline clustering model specific to the device;
detect anomalous traffic behavior of the device by identifying a second data point in the at least one sample as an anomalous data point based on the risk factor score of that second data point and that second data point being a determined outlier;
update the plurality of clusters of the baseline clustering model based on the at least one sample and the risk factor score for each second data point of each sample, wherein the update includes adding each sample having a high risk factor score as a new cluster to the plurality of clusters; and
performing at least one mitigation action based on the detected anomalous traffic behavior.
12. The system of claim 11 , wherein the system is further configured to:
for each second data point of each sample, determine whether the second data point is an outlier with respect to one of the plurality of clusters, wherein the anomalous traffic behavior is detected further based on at least one outlier second data point of the plurality of second data points.
13. The system of claim 11 , wherein the risk factor score for each second data point in the plurality of second data points is determined based on a vector representation of the second data point in a traffic feature space and a proximity of the second data point to one of the plurality of clusters.
14. The system of claim 11 , wherein the system is further configured to:
integrate each sample having a risk factor score below a threshold into one of the plurality of clusters.
15. The system of claim 11 , wherein the system is further configured to:
compare the risk factor score of each second data point of each sample to a threshold thereby determining whether the respective second data point for the risk factor score is high risk, wherein the respective second data point is high risk when the risk factor score for the respective second data point is above the threshold.
16. The system of claim 15 , wherein the system is further configured to:
update the threshold based on the detected anomalous traffic behavior.
17. The system of claim 15 , wherein the threshold is a proportional value of a highest risk factor score among each second data point of each sample.
18. The system of claim 13 , wherein the risk factor score is determined further based on a norm of the vector representation of the second data point, a length of a projection of the vector representation over a centroid of closest cluster of the plurality of clusters, and a length of the projection over a diagonal vector of the traffic feature space.