System and method for detection of denial of service attacks
Systems and methods for detecting a denial of service attack are disclosed. These may include receiving a plurality of web log traces from one of a plurality of web servers; extracting a first set of features from the plurality of web log traces; applying a first machine learning technique to the first set of features; producing a first plurality of user classifications for communication to the web server; extracting a second set of features from the plurality of web log traces; applying a second machine learning technique to the second set of features; producing a second plurality of user classification for communication to the web server; communicating the first plurality of user classifications to the web server based at least on the plurality of web log traces; and communicating the second plurality of user classifications to the web server based at least on the plurality of web log traces.
1. A distributed denial of service (“DDOS”) detection engine communicatively coupled to a plurality of web servers, the DDOS detection engine comprising:
a web server interface configured to:
receive a plurality of web log traces from a web server, the web server being one of the plurality of web servers;
communicate a first plurality of user classifications to the web server based at least on the plurality of web log traces; and
communicate a second plurality of user classifications to the web server based at least on the plurality of web log traces; and
a first DDOS analysis engine configured to:
extract a first feature vector from the plurality of web log traces, wherein the first feature vector is representative of network traffic on the plurality of web servers over a first period of time;
apply a first machine learning technique to the first feature vector; and
produce the first plurality of user classifications for communication to the web server in substantially real time; and
a second DDOS analysis engine configured to:
extract a second feature vector from the plurality of web log traces, wherein the second feature vector is representative of network traffic on the plurality of web servers over a second period of time, the second period of time greater than the first period of time;
apply a second machine learning technique to the second feature vector; and
produce the second plurality of user classification for communication to the web server.
2. The DDOS detection engine of claim 1 , wherein the DDOS detection engine further comprises a DDOS alert module, the DDOS alert module configured to alert the plurality of web servers of a DDOS attack on the web server.
3. The DDOS detection engine of claim 1 , wherein the first machine learning technique comprises an entropy analysis of the plurality of web log traces.
4. The DDOS detection engine of claim 1 , wherein the second machine learning technique comprises a random forest analysis of the plurality of web log traces.
5. The DDOS detection engine of claim 1 , wherein the second machine learning technique comprises a support vector machine analysis of the plurality of web log traces.
6. The DDOS detection engine of claim 1 , wherein the web server interface is further configured to communicate the first plurality of user classifications to the plurality of web servers in substantially real time.
7. The DDOS detection engine of claim 1 , wherein the first feature vector comprises a source internet protocol identifier.
8. The DDOS detection engine of claim 1 , wherein the first feature vector comprises a uniform resource indicator.
9. The DDOS detection engine of claim 1 , wherein the first feature vector comprises a referrer indicator.
10. The DDOS detection engine of claim 1 , wherein the first feature vector comprises a user agent indicator.
11. The DDOS detection engine of claim 1 , wherein the first feature vector comprises a source internet protocol identifier.
12. The DDOS detection engine of claim 11 , wherein the first feature vector further comprises a total number of GET requests sent by the source internet protocol identifier.
13. The DDOS detection engine of claim 11 , wherein the first feature vector further comprises a total number of POST requests sent by the source internet protocol identifier.
14. The DDOS detection engine of claim 11 , wherein the first feature vector further comprises a total number of requests sent by the source internet protocol identifier that are neither GET nor POST requests.
15. The DDOS detection engine of claim 11 , wherein the first feature vector further comprises a list of requests sent by the source internet protocol identifier that return a standardized status code.
16. The DDOS detection engine of claim 15 , wherein the first feature vector further comprises a plurality of ratios comparing a total of a single standardized status code returned from requests sent by the source internet protocol identifier to the list of requests sent by the source internet protocol identifier that return the standardized status code.
17. The DDOS detection engine of claim 11 , wherein the first feature vector further comprises a statistical measure of time taken for a request sent by the source internet protocol identifier.
18. The DDOS detection engine of claim 11 , wherein the first feature vector further comprises a statistical measure of size of a request sent by the source internet protocol identifier.
19. The DDOS detection engine of claim 11 , wherein the first feature vector further comprises a statistical measure of session activity for a request sent by the source internet protocol identifier.
20. The DDOS detection engine of claim 11 , wherein the first feature vector further comprises a statistical measure of a probability of the source internet protocol identifier requesting a requested uniform resource.
21. A method for detecting a distributed denial of service (“DDOS”) attack on a networked system comprising a plurality of web servers, the method comprising:
receiving a plurality of web log traces from a web server, the web server being one of the plurality of web servers;
extracting a first feature vector from the plurality of web log traces, wherein the first feature vector is representative of network traffic on the plurality of web servers over a first period of time;
applying a first machine learning technique to the first feature vector;
producing a first plurality of user classifications for communication to the web server in substantially real time;
extracting a second feature vector from the plurality of web log traces, wherein the second feature vector res is representative of network traffic on the plurality of web servers over a second period of time, the second period of time greater than the first period of time;
applying a second machine learning technique to the second feature vector;
producing a second plurality of user classification for communication to the web server;
communicating the first plurality of user classifications to the web server based at least on the plurality of web log traces; and
communicating the second plurality of user classifications to the web server based at least on the plurality of web log traces.
22. The method of claim 21 , further comprising alerting the plurality of web servers of a DDOS attack on the web server.
23. The method of claim 21 , wherein the first machine learning technique comprises an entropy analysis of the plurality of web log traces.
24. The method of claim 21 , wherein the second machine learning technique comprises a random forest analysis of the plurality of web log traces.
25. The method of claim 21 , wherein the second machine learning technique comprises a support vector machine analysis of the plurality of web log traces.
26. The method of claim 21 , wherein communicating the first plurality of user classifications to the plurality of web servers occurs in substantially real time.
27. The method of claim 21 , wherein the first feature vector comprises a source internet protocol identifier.
28. The method of claim 21 , wherein the first feature vector comprises a uniform resource indicator.
29. The method of claim 21 , wherein the first feature vector comprises a referrer indicator.
30. The method of claim 21 , wherein the first feature vector comprises a user agent indicator.
31. The method of claim 21 , wherein the first feature vector comprises a source internet protocol identifier.
32. The method of claim 31 , wherein the first feature vector further comprises a total number of GET requests sent by the source internet protocol identifier.
33. The method of claim 31 , wherein the first feature vector further comprises a total number of POST requests sent by the source internet protocol identifier.
34. The method of claim 31 , wherein the first feature vector further comprises a total number of requests sent by the source internet protocol identifier that are neither GET nor POST requests.
35. The method of claim 31 , wherein the first feature vector further comprises a list of requests sent by the source internet protocol identifier that return a standardized status code.
36. The method of claim 35 , wherein the first feature vector further comprises a plurality of ratios comparing a total of a single standardized status code returned from requests sent by the source internet protocol identifier to the list of requests sent by the source internet protocol identifier that return the standardized status code.
37. The method of claim 31 , wherein the first feature vector further comprises a statistical measure of time taken for a request sent by the source internet protocol identifier.
38. The method of claim 31 , wherein the first feature vector further comprises a statistical measure of size of a request sent by the source internet protocol identifier.
39. The method of claim 31 , wherein the first feature vector further comprises a statistical measure of session activity for a request sent by the source internet protocol identifier.
40. The method of claim 31 , wherein the first feature vector further comprises a statistical measure of a probability of the source internet protocol identifier requesting a requested uniform resource.