Assigning outlier-related classifications to traffic flows across multiple time windows
View Patent ↗Systems and methods are provided for combining a multiple sub-time window sampling architecture with machine learning to detect outlier traffic flow behavior which may indicate malicious/problematic network activity. For example, a network device may obtain a sample of traffic flow data during a defined time window. The sample of traffic flow data may comprise information associated with a sampled subset of traffic flows transferred by a network device in the defined time window. The network device may partition the defined time window into two or more sub-time windows. In each sub-time window, using machine learning, the network device may assign an outlier-related classification to each sampled traffic flow based on the relative behavioral characteristics of all the sampled traffic flows. The network device may aggregate the outlier-related classifications for each sampled traffic flow across multiple sub-time windows, and process traffic flows based on the aggregated outlier-related classifications.
1. A method comprising:
obtaining a sample of traffic flow data during a defined time window, wherein:
the sample of traffic flow data comprises information associated with a sampled subset of all traffic flows transferred by a network device in the defined time window, and
the sampled traffic flows have one or more behavioral characteristics;
partitioning the defined time window into two or more sub-time windows;
in at least one sub-time window, using an outlier classification machine learning algorithm to assign an outlier-related classification to one or more of the sampled traffic flows based on relative behavioral characteristics of the sampled traffic flows; and
aggregating the outlier-related classifications for one or more sampled traffic flows during the defined time window.
2. The method of claim 1 , further comprising, processing traffic flows based on the aggregated outlier-related classifications for one or more sampled traffic flows.
3. The method of claim 2 , further comprising, defining a subset of the sampled traffic flows as top outliers for the defined time window based on the aggregated outlier-related classifications for one or more sampled traffic flows.
4. The method of claim 2 , wherein processing traffic flows based on the aggregated outlier-related classifications for one or more sampled traffic flows comprises providing information associated with the top outliers to a user interface.
5. The method of claim 2 , wherein the one or more behavioral characteristics comprise at least one of the following:
data packet volume;
mean data packet length;
median data packet length;
minimum data packet length;
maximum data packet length; and
total number of data packets.
6. The method of claim 2 , wherein one or more of the sampled traffic flows also comprise one or more identifying characteristics, the identifying characteristics comprising at least one of the following:
source Internet Protocol (IP);
destination IP;
source port;
destination port; and
IP protocol.
7. The method of claim 2 , wherein the two or more sub-time windows are equal in length, and are each ten minutes or less.
8. The method of claim 2 , wherein the length of the two or more sub-times windows is determined using a sub-time window length determining machine learning algorithm.
9. The method of claim 8 , wherein the length of the two or more sub-time windows maximizes the number of sampled traffic flows which are classified as outliers per sub-time window.
10. The method of claim 8 , wherein the outlier classification machine learning algorithm is an unsupervised machine learning algorithm.
11. A network device comprising at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
obtaining a sample of traffic flow data during a defined time window, wherein:
the sample of traffic flow data comprises information associated with a sampled subset of all traffic flows transferred by the network device in the defined time window, and
each sampled traffic flow has one or more behavioral characteristics;
partitioning the defined time window into two or more sub-time windows;
in at least one sub-time window, using an outlier classification machine learning algorithm to assign an outlier-related classification to one or more sampled traffic flows based on relative behavioral characteristics of the sampled traffic flows;
aggregating the outlier-related classifications for one or more sampled traffic flows during the defined time window;
defining a subset of the sampled traffic flows as top outliers for the defined time window based on the aggregated outlier-related classifications for one or more sampled traffic flows; and
processing traffic flows based on the defined subset of top outliers.
12. The network device of claim 11 , wherein processing traffic flows based on the defined subset of top outliers comprises at least one of the following:
providing information associated with the top outliers to a user interface;
reauthenticating the top outliers; and
re-routing the top outliers to a firewall.
13. The network device of claim 12 , wherein the one or more behavioral characteristics comprise at least one of the following:
data packet volume;
mean data packet length;
median data packet length;
minimum data packet length;
maximum data packet length; and
total number of data packets.
14. The network device of claim 13 , wherein each sampled traffic flow also comprises one or more of the following identifying characteristics:
source Internet Protocol (IP);
destination IP;
source port;
destination port; and
IP protocol.
15. The network device of claim 11 , wherein the two or more sub-time windows are equal in length, and are each ten minutes or less.
16. The network device of claim 15 , wherein the length of the two or more sub-time windows is determined using a sub-time window length determining machine learning algorithm.
17. The network device of claim 16 , wherein the length of the two or more sub-time windows maximizes the number of sampled traffic flows which are classified as outliers per sub-time window.
18. The network device of claim 16 , wherein the outlier classification machine learning algorithm is an unsupervised machine learning algorithm.
19. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
obtaining a sample of traffic flow data during a defined time window, wherein:
the sample of traffic flow data comprises information associated with a sampled subset of all traffic flows transferred by a network device in the defined time window, and
each sampled traffic flow has one or more behavioral characteristics;
partitioning the defined time window into two or more sub-time windows;
in each sub-time window, using an outlier classification machine learning algorithm to assign an outlier-related classification to one or more sampled traffic flows based on relative behavioral characteristics of the sampled traffic flows;
aggregating the outlier-related classifications for each sampled traffic flow during the defined time window;
processing traffic flows based on the aggregated outlier-related classifications for one or more sampled traffic flows.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the lengths of the two or more sub-time windows are determined using a sub-time window length determining machine learning algorithm.