Methods and systems for network flow analysis
View Patent ↗A system and method comprising: receiving network flow data; identifying a peer to peer network flow within the network flow data comparing the peer to peer network flow to a known peer to peer application flow; labeling the peer to peer network flow as the known peer to peer application flow when the peer to peer network flow matches the known peer to peer application flow; and creating a data set to be associated with the labeled peer to flow.
1. A method comprising:
performing processing associated with receiving, with a flow creation module in communication with a computer comprising a database, network flow data;
performing processing associated with identifying, with a peer to peer flow detection module in communication with the computer, and from the network flow data, a first plurality of network flows that together constitute a first peer to peer network communication within the network flow data and a second plurality of network flows that together constitute a second peer to peer network communication within the network flow data;
performing processing associated with detecting, with a peer to peer classification module in communication with the computer, that the first plurality of network flows matches one or more known peer to peer application communications;
responsive to detecting a match for the first plurality of network flows, performing processing associated with labeling, with the peer to peer classification module, the first plurality of network flows with a first label comprising a category identical to the matching one or more known peer to peer application communications;
performing processing associated with detecting, with the peer to peer classification module in communication with the computer, that the second plurality of network flows does not match any of the one or more known peer to peer application communications;
responsive to failing to detect a match for the second plurality of network flows:
performing processing associated with determining, with the peer to peer classification module, that one or more connection features for the second plurality of network flows resemble one or more connection features for a stored unclassified peer to peer application communication;
responsive to determining that the one or more connection features for the second plurality of network flows resemble the one or more connection features for the stored unclassified peer to peer application communication, performing processing associated with clustering, with an unclassified peer to peer clustering module, the second plurality of network flows with the stored unclassified peer to peer application communication; and
performing processing associated with labeling, with the peer to peer classification module, the second plurality of network flows with a second label based on its cluster;
performing processing associated with determining, with the peer to peer classification module, whether the first plurality of network flows are malicious based on the category;
performing processing associated with determining, with the peer to peer classification module, whether the second plurality of network flows are malicious based on one or more characteristics of the stored unclassified peer to peer application communication; and
in response to determining that one or more of the first plurality of network flows and the second plurality of network flows are malicious, performing processing associated with generating, with an alert module in communication with the computer, an alert and blocking at least one of the first plurality of network flows or at least one of the second plurality of network flows.
2. The method of claim 1 , further comprising:
performing processing associated with identifying, with the peer to peer flow detection module in communication with the computer, and from the network flow data, a third plurality of network flows that together constitute a third peer to peer network communication within the network flow data,
performing processing associated with labeling, with the peer to peer classification module, the third plurality of network flows as an unclassified peer to peer application communication when the third plurality of network flows do not resemble the stored unclassified peer to peer application communication.
3. The method of claim 1 , further comprising:
performing processing associated with receiving, with a peer to peer identifier assignment module in communication with the computer, the second label for the peer to peer application cluster, the second label further defining the peer to peer application cluster as a known peer to peer application communication.
4. The method of claim 1 , wherein performing the processing associated with receiving the network flow data comprises performing processing associated with creating the network flow data from packet information.
5. The method of claim 1 , wherein the network flow data is received from a router.
6. The method of claim 1 , wherein performing the processing associated with identifying first the plurality of network flows within the network flow data comprises performing processing associated with comparing the network flow data to a white-listed known peer to peer application communication and ignoring the network flow data corresponding to the first plurality of network flows when it matches the white-listed known peer to peer application communication.
7. The method of claim 1 , further comprising:
performing processing associated with examining the network flow data to determine at least one of:
whether a recent DNS lookup has been performed in relation to a remote computer associated with the second plurality of network flows, and
whether the second plurality of network flows are associated with a high activity domain;
and
performing processing associated with ignoring the second plurality of network flows when at least one of the following is true:
a recent DNS lookup has been performed, and
the second plurality of network flows is associated with the high activity domain.
8. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communications comprises performing processing associated with:
iteratively analyzing the first plurality of network flows over a plurality of epochs; and
comparing a characteristic of the first plurality of network flows observed during one of the plurality of epochs to a characteristic of the known peer to peer application communication;
wherein each of the plurality of epochs is a different length of time from each of the remaining plurality of epochs.
9. The method of claim 8 , further comprising performing processing associated with ignoring the plurality of network flows when the first plurality of network flows are older than a longest of the plurality of epochs.
10. The method of claim 1 , wherein the second plurality of network flows comprise a merged first peer to peer network flow and a second peer to peer network flow related to the first peer to peer network flow.
11. The method of claim 10 , wherein a P2P faux-session creation module is used to merge the first peer to peer network flow and the second peer to peer network flow.
12. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communications comprises performing processing associated with generating a feature of at least one of the first plurality of network flows, the feature comprising at least one of: a payload out, a payload in, a packet out, and a packet in.
13. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communications comprises:
examining a host involved with the first plurality of network flows to determine how the host has participated in a previous peer to peer application flow;
determining whether the first plurality of network flows is associated with at least one of a private network, a public network, and a public/private network; and
biasing the comparison of the first plurality of network flows to the previous peer to peer application flow when the previous peer to peer application flow resembles the first plurality of network flows.
14. The method of claim 1 , wherein performing the processing associated with receiving the second label for the peer to peer application cluster comprises performing processing associated with generating the second label.
15. The method of claim 1 , wherein performing the processing associated with receiving the second label for the peer to peer application cluster comprises performing processing associated with receiving the second label from a user input.
16. The method of claim 1 , further comprising performing processing associated with displaying, with a display, the alert.
17. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communications comprises performing processing associated with generating a feature of at least one of the first plurality of network flows, the feature comprising at least one of: a transmission control protocol (TCP) flag, a user datagram protocol (UDP) flag, an internet control message protocol (ICMP) flag, a high port—high port flag, a high port—low port flag, a low port—low port flag, an internal TCP control port flag, an external TCP control port flag, an internal UDP control port flag, and an external UDP control port flag.
18. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communications comprises performing processing associated with generating a feature of at least one of the first plurality of network flows, the feature comprising at least one of: an average packet exchange rate, average active flows, maximum active simultaneous flows, and maximum external hosts.
19. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communications comprises performing processing associated with generating a feature of at least one of the first plurality of network flows, the feature comprising at least one of: total flows, a session duration, an average flow duration, maximum simultaneous data transfers, a one to one transfers flag, a many to one transfers flag, a percent of one to one transfers to total transfers, and a number of persistent flows.
20. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communications comprises performing processing associated with generating a feature of at least one of the first plurality of network flows, the feature comprising at least one of: a percent of persistent flows, a percent of one-way flows, a percent of low traffic flows, a percent of medium traffic flows, a percent of high traffic flows.
21. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communications comprises performing processing associated with generating a feature of at least one of the first plurality of network flows, the feature comprising at least one of: a possible voice flag, a possible internet radio flag, a possible audio share flag, and a possible video share flag.
22. The method of claim 1 , wherein performing the processing associated with detecting that the first plurality of network flows matches one or more known peer to peer application communication comprises performing processing associated with generating a feature of at least one of the first plurality of network flows, the feature comprising at least one of: a geographic dispersion, and an NXDOMAIN response.