Statistics collection for network traffic
View Patent ↗A system architecture for thwarting denial of service attacks on a victim data center is described. The system includes a first plurality of monitors that monitor network traffic flow through the network. The first plurality of monitors is disposed at a second plurality of points in the network. The system includes a central controller that receives data from the plurality of monitors, over a hardened, redundant network. The central controller analyzes network traffic statistics to identify malicious network traffic. In some embodiments of the system, a gateway device is disposed to pass network packets between the network and the victim site. The gateway is disposed to protect the victim site, and is coupled to the control center by the redundant hardened network.
1. A machine implemented method of monitoring traffic flow in a monitoring device disposed to receive network traffic packets, the method comprising:
producing statistics corresponding to a parameter of traffic flow to trace the source of an attack, with producing further comprising:
mapping the traffic flow into a plurality of buckets by applying a hash function “f(h)” to the parameter of the traffic flow to output an integer corresponding to one of the buckets;
accumulating statistics from the packets; and
comparing the number of buckets to a threshold; and
adjusting the number of buckets based on determining whether the number of buckets should be divided into more buckets or combined into fewer buckets based on comparing the number of buckets to the threshold.
2. The method of claim 1 wherein the buckets are storage areas in a memory space of the monitor device.
3. The method of claim 1 wherein as the number of buckets changes, the buckets have values derived from the buckets prior to the change.
4. The method of claim 1 wherein the hash function adapts to map to the new number of buckets, as the new number of buckets changes.
5. The method of claim 1 wherein comparing statistic values comprises:
comparing the value accumulated in the bucket to a threshold that depends on the number of buckets.
6. The method of claim 1 wherein the parameter is the count of how many packets a data collector or gateway examines.
7. The method of claim 1 wherein as a value of a parameter for one bucket approaches a threshold, the monitoring device raises an alarm.
8. The method of claim 1 wherein the hash function changes periodically in a randomly secret manner so that packets are reassigned to different buckets.
9. The method of claim 1 wherein the variable number of buckets dynamically adjusts the amount of traffic and number of flows monitored, so that the monitoring device is not vulnerable to a denial of service attack against its own resources.
10. The method of claim 1 wherein the variable number of buckets efficiently identifies the source or sources of attack by breaking down traffic into different buckets and examining statistics accumulated for a parameter and a corresponding threshold in each bucket.
11. The method of claim 1 wherein the traffic is monitored at multiple levels of granularity, from aggregate to individual flows.
12. The method of claim 1 wherein the method is applied to monitoring of TCP packet ratios and repressor traffic.
13. The method of claim 1 wherein the threshold is a first threshold and the method further comprises:
comparing accumulated statistic values from the buckets to second threshold values to determine that an event is of significance.
14. A computer program product residing on a computer readable storage medium for monitoring network traffic flow in a network, the computer program product comprising instructions for causing a computer to:
map, on a computer, traffic flow into a plurality of buckets by applying a hash function “f(h)” to a parameter of the traffic flow to output an integer corresponding to one of the buckets;
accumulate statistics from the packets; and
compare the accumulated statistic values from the buckets to configured threshold values corresponding to the number of buckets to determine that an event is of significance; and
adjust the number of buckets as the number of buckets approaches a second threshold.
15. The computer program product of claim 14 wherein based on the second threshold, the buckets are divided into more buckets or combined into fewer buckets.
16. The computer program product of claim 14 wherein instructions to monitor further comprise instructions to
divide the bucket into a different number of new buckets containing values derived from the original bucket.
17. The computer program product of claim 14 wherein the hash function adapts to map to the new number of buckets as the new number of buckets changes.
18. The computer program product of claim 14 wherein the parameter is the count of how many packets a data collector or gateway examines.
19. The computer program product of claim 14 wherein the buckets are storage areas in the memory space of the monitor device.
20. The computer program product of claim 14 wherein the hash function changes periodically a randomly secret manner so that packets are reassigned to different buckets.
21. A data collector to collect statistical information about network flows comprising:
a computer read ante medium;
a computing device with a processor that executes a computer program product stored on the computer readable medium comprising instructions to cause the computing device to:
map traffic flow into a plurality of buckets by applying a hash function “f(h)” to the parameter of the traffic flow to output an integer corresponding to one of the buckets;
accumulate statistics from the packets; and
compare the accumulated statistic values from the buckets to configured threshold values corresponding to the number of buckets to determine that an event is of significance; and
adjust the number of buckets as the number of buckets approaches a second threshold.
22. The data collector of claim 21 wherein based on the second threshold, the buckets are divided into more buckets or combined into fewer buckets.
23. The data collector of claim 21 wherein instructions to monitor further comprises instructions to
divide the bucket into a different number of new buckets containing values, derived from the original bucket.
24. The data collector of claim 21 wherein the hash function adapts to map to the new number of buckets as the new number of buckets changes.
25. The data collector of claim 21 wherein the parameter is the count of how many packets the data collector examines.
26. The data collector of claim 21 wherein the buckets are storage areas in the memory space of the monitor device.
27. The data collector of claim 21 wherein the hash function changes periodically in a randomly secret manner so that packets are reassigned to different buckets.
28. The data collector of claim 21 wherein instructions to compare statistic values comprises instructions to:
compare the value accumulated in the bucket to a threshold that depends on the number of buckets.
29. The data collector of claim 21 wherein as a value of a parameter for one bucket approaches a threshold, the monitoring device raises an alarm.
30. The data collector of claim 21 wherein the variable number of buckets dynamically adjusts the amount of traffic and number of flows monitored, so that the data collector is not vulnerable to a denial of service attack against its own resources.
31. The data collector of claim 21 wherein the variable number or buckets efficiently identities the source or sources of attack by breaking down traffic into different buckets and examining statistics accumulated for a parameter and a corresponding threshold in each bucket.
32. The data collector of claim 21 wherein the traffic is monitored at multiple levels of granularity, from aggregate to individual flows.
33. The data collector of claim 21 wherein the traffic is applied to monitoring of TCP packet ratios and repressor traffic.
34. The data collector of claim 21 wherein the threshold is a first threshold and the computer program further comprises instructions to:
compare accumulated statistic values from the buckets to second threshold values to determine that an event is of significance.
35. The data collector of claim 21 further comprising:
a port to link the data collector to a central control center.
36. A method of monitoring traffic flow in a monitor device disposed to receive network packets, the method comprising:
producing statistics corresponding to a parameter of the traffic flow to trace a source of an attack, with producing further comprising:
mapping the traffic flow into a plurality of buckets;
varying the number of buckets according to the amount of traffic and number of flows to breakdown traffic, flow into different buckets; and
analyzing statistics accumulated for a parameter and a corresponding threshold in the bucket to identify the source of the attack.
37. The method of claim 35 wherein varying varies the number of buckets so that the monitor device is not vulnerable to DoS attacks against its own resources.
38. The method of claim 35 wherein varying the number of buckets comprises:
comparing the number of buckets to a threshold number or buckets;
determining whether the number of buckets should be divided into more buckets or combined into fewer buckets based on comparing the number of buckets to the threshold and as the number of buckets changes, the buckets have values derived from the buckets prior to the change.
39. The method of claim 36 wherein further comprising:
comparing accumulated statistic values from the buckets to second threshold values to determine that an event is of significance.
40. The method of claim 36 wherein comparing statistic values comprises:
accumulating statistic values from the packets; and
comparing the values accumulated in the buckets to thresholds that depend on the number of buckets.
41. The method of claim 36 wherein the variable number of buckets dynamically adjusts the amount of traffic and number or flows monitored, so that the monitoring device is not vulnerable to a denial of service attack against its own resources.
42. The method of claim 36 wherein the buckets are storage areas in a memory space of the monitor device and mapping the traffic flow into a plurality of buckets comprises:
applying a hash function “f(h)” to the parameter of the traffic flow to output an integer corresponding to one of the buckets.
43. A computer program product reaming on a computer readable medium for monitoring traffic flow in a monitor device disposed to receive network packets, the commuter program product comprises instructions for causing the device to:
produce, on a computer, statistics corresponding to a parameter of the traffic flow to trace a source of an attack, with producing further comprising:
map the traffic flow into a plurality of buckets;
vary the number of buckets according to the amount of traffic and number of flows to breakdown the traffic flow into different buckets; and
analyze statistics accumulated for a parameter and a corresponding threshold in the bucket to identify a source of the attack.
44. The computer program product of claim 43 wherein instructions to vary, vary the number of buckets so that the monitoring device is not vulnerable to DoS attacks against its own resources.
45. The computer program product of claim 43 wherein instructions to vary comprises instructions to:
compare the number of buckets to a threshold number of buckets;
determine whether the number of buckets should be divided into more buckets or combined into fewer buckets based on comparing the number of buckets to the threshold and as the number of buckets changes, the buckets have values derived from the buckets prior to the change.
46. The computer program product of claim 43 further comprising instructions to:
compare accumulated statistic values from the buckets to second threshold values to determine that an event is of significance.
47. The computer program product of claim 43 wherein instructions to compare statistic values comprises instructions to:
accumulate statistic values from the packets; and
compare the values accumulated in the buckets to thresholds that depend on the number of buckets.
48. The computer program product of claim 43 wherein the variable number of buckets dynamically adjusts the amount of traffic and number of flows monitored, so that the monitoring device is not vulnerable to a denial of service attack against its own resources.
49. The computer program product of claim 43 wherein the buckets are storage areas in a memory space of the monitor device and instructions to map the traffic flow into a plurality of buckets comprises instructions to:
apply a hash function “f(h)” to the parameter of the traffic flow to output an integer corresponding to one of the buckets.