System and method for sampling network traffic
Disclosed herein are systems, computer-implemented methods, and computer-readable media for sampling network traffic. The method includes receiving a plurality of flow records, calculating a hash for each flow record based on one or more invariant part of a respective flow, generating a quasi-random number from the calculated hash for each respective flow record, and sampling flow records having a quasi-random number below a probability P. Invariant parts of flow records include destination IP address, source IP address, TCP/UDP port numbers, TCP flags, and network protocol. A plurality of routers can uniformly calculate hashes for flow records. Each router in a plurality of routers can generate a same quasi-random number for each respective flow record and uses different values for probability P. The probability P can depend on a flow size. The method can divide the quasi-random number by a maximum possible hash value.
1. A computer-implemented method of sampling network traffic, the method comprising:
receiving a plurality of flow records;
calculating a hash for each flow record based on one or more invariant part of a respective flow record;
generating a quasi-random number from the calculated hash for each respective flow record; and
sampling flow records having a quasi-random number below a probability P.
2. The computer-implemented method of claim 1 , wherein the one or more invariant part of each flow record includes destination IP address, source IP address, TCP/UDP port numbers, TCP flags, and network protocol.
3. The computer-implemented method of claim 1 , wherein a plurality of routers uniformly calculates hashes for flow records.
4. The computer-implemented method of claim 1 , wherein each router in a plurality of routers generates a same quasi-random number for each respective flow record and uses different values for probability P.
5. The computer-implemented method of claim 1 , wherein each router in a plurality of routers uses a same value for probability P.
6. The computer-implemented method of claim 1 , wherein the probability P depends on a field in the respective flow record.
7. The computer-implemented method of claim 1 , the method further comprising dividing the quasi-random number by a maximum possible hash value to obtain a number between 0 and 1 inclusive.
8. The computer-implemented method of claim 1 , the method further comprising exporting sampled flow records to a central collector.
9. A system for sampling network traffic, the system comprising:
a processor;
a first module configured to control the processor to receive a plurality of flow records;
a second module configured to control the processor to calculate a hash for each flow record based on one or more invariant part of a respective flow record;
a third module configured to control the processor to generate a quasi-random number from the calculated hash for each respective flow record; and
a fourth module configured to control the processor to sample flow records having a quasi-random number below a probability P.
10. The system of claim 9 , wherein the one or more invariant part of each flow record includes destination IP address, source IP address, TCP/UDP port numbers, TCP flags, and network protocol.
11. The system of claim 9 , wherein a plurality of routers uniformly calculates hashes for flow records.
12. The system of claim 9 , wherein each router in a plurality of routers generates a same quasi-random number for each respective flow record and uses different values for probability P.
13. The system of claim 9 , wherein each router in a plurality of routers uses a same value for probability P.
14. The system of claim 9 , wherein the probability P depends on a field in the respective flow record.
15. The system of claim 9 , further comprising a fifth module configured to control the processor to divide the quasi-random number by a maximum possible hash value to obtain a number between 0 and 1 inclusive.
16. The system of claim 9 , the comprising a fifth module configured to control the processor to export sampled flow records to a central collector.
17. A non-transitory computer-readable medium storing a computer program having instructions for sampling network traffic, the instructions comprising:
receiving a plurality of network flow records;
calculating a hash for each flow record based on one or more invariant part of a respective flow;
generating a quasi-random number from the calculated hash for each respective flow record; and
sampling flow records having a quasi-random number below a probability P.
18. The non-transitory computer-readable medium of claim 17 , wherein the one or more invariant part of each flow record includes destination IP address, source IP address, TCP/UDP port numbers, TCP flags, and network protocol.
19. The non-transitory computer-readable medium of claim 17 , wherein a plurality of routers uniformly calculates hashes for flow records.
20. The non-transitory computer-readable medium of claim 17 , wherein each router in a plurality of routers generates a same quasi-random number for each respective flow record and uses different values for probability P.