Webcrawl internet security analysis and process

1. A process of detecting security vulnerabilities present in a target Web site, comprising the steps of:

establishing an Internet connection with the target Web site;

retrieving a Web page for the target Web site;

parsing through the Web page to identify any linked-to Web pages or objects which are included in the Web page; and

identifying security vulnerabilities by performing at least one of the steps of (i) scanning the target Web site for at least one vulnerability and (ii) applying at least one hack method to the target Web site.

2. The process of claim 1 wherein the step of identifying security vulnerabilities comprises both of the steps (i) scanning the target Web site for at least one vulnerability and (ii) applying at least one hack method to the target Web site.

3. The process of claim 1 further comprising automatically passing an authorized username and password to the target Web site, if required to gain access to the target Web site.

4. The process of claim 1 , further comprising scanning at least one of the security vulnerabilities for at least one exploit in order to identify further security vulnerabilities.

5. The process of claim 1 , further comprising parsing through the linked-to Web pages to identify any further-linked-to Web pages or objects which are included in the linked-to Web pages.

6. The process of claim 5 , further comprising parsing through the first mentioned Web page to identify any hidden Web pages or objects which are included in the hidden Web pages.

7. The process of claim 6 , further comprising parsing through the hidden Web-pages to identify any further-hidden Web pages or objects which are included in the further-hidden Web pages.

8. The process of claim 7 , further comprising:

comparing each hidden Web page and each further-hidden Web page to each linked-to Web page and each further-linked-to Web page; and

identifying each hidden Web page and each further-hidden Web page that is different from the linked-to Web pages and the further-linked to Web pages.

9. The process of claim 6 , wherein the step of identifying security vulnerabilities comprises applying at least one hack method and includes attempting to access unauthorized files located outside the target Web site's root directory.

10. The process of claim 9 , wherein the step of identifying security vulnerabilities comprises scanning the target Web site for at least one vulnerability and includes checking for at least one common filename.

11. The process of claim 10 , wherein the at least one common filename is selected from the group consisting of “msadcs.dil” and “WS_FTP.LOG.”

12. The process of claim 6 , wherein the step of identifying security vulnerabilities comprises applying at least one hack method and includes automatically passing multiple usernames and passwords to the target Web site if a login Web page is encountered.

13. The process of claim 5 , wherein the parsing through the first mentioned Web page and the parsing through the linked-to Web pages include performing a keyword search in order to detect at least one point of interest.

14. The process of claim 13 , wherein the at least one point of interest is selected from the group consisting of an administration Web page and a directory list tag.

15. The process of claim 14 , wherein the step of identifying security vulnerabilities comprises applying at least one hack method and includes attempting to access unauthorized files located outside the target Web site's root directory.

16. A process of detecting security vulnerabilities present in a target Web site, comprising the steps of:

establishing an Internet connection with the target Web site;

retrieving a Web page for the target Web site;

parsing through the Web page to identify any linked-to Web pages or objects which are included in the Web page, wherein the parsing includes performing a keyword search in order to detect at least one point of interest;

identifying security vulnerabilities by performing at least one of the steps of (i) scanning the target Web site for at least one vulnerability and (ii) applying at least one hack method to the target Web site; and

prioritizing the security vulnerabilities.

17. The process of claim 16 wherein the step of identifying security vulnerabilities comprises both of the steps (i) scanning the target Web site for at least one vulnerability and (ii) applying at least one hack method to the target Web site.

18. The process of claim 17 , wherein the applying of at least one predetermined hack method includes attempting to access unauthorized files located outside the target Web site's root directory.

19. The process of claim 17 , further comprising automatically passing an authorized username and password to the target Web site, if required to gain access to the target Web site.

20. The process of claim 17 , wherein the applying of at least one hack method includes automatically passing multiple usernames and passwords to the target Web site if a login Web page is encountered.

21. The process of claim 17 , wherein the applying of at least one hack method includes passing invalid data to a data entry field of the target Web site and evaluating the result.

22. The process of claim 21 , further comprising:

recording the invalid data which produces a security vulnerability; and

passing the recorded invalid data to at least one other data entry field of the target Web site.

23. The process of claim 16 , further comprising parsing through the Web page to identify any hidden Web pages or objects which are included in the hidden Web pages.

24. The process of claim 23 , further comprising parsing through the hidden Web-pages to identify any further-hidden Web pages or objects which are included in the further-hidden Web pages.

25. The process of claim 24 , further comprising:

comparing each hidden Web page and each further-hidden Web page to each linked-to Web page and each further-linked-to Web page; and

identifying each hidden Web page and each further-hidden Web page that is different from the linked-to Web pages and the further-linked to Web pages.

26. The process of claim 16 , further comprising parsing through the linked-to Web pages to identify any further-linked-to Web pages or objects which are included in the linked-to Web pages.

27. The process of claim 26 , wherein the at least one point of interest is selected from the group consisting of an administration Web page and a directory list tag.

28. The process of claim 26 , further comprising scanning at least one of the security vulnerabilities for at least one exploit in order to identify further security vulnerabilities.

29. A process of detecting security vulnerabilities present in a target Web site, comprising the steps of:

establishing an Internet connection with the target Web site;

retrieving a Web page for the target Web site;

parsing through the Web page to identify any linked-to Web pages or objects which are included in the default Web page;

scanning the target Web site for at least one exploit in order to identify security vulnerabilities;

applying at least one hack method to the target Web site in order to identify security vulnerabilities, wherein the applying at least one hack method includes attempting to access unauthorized files located outside the target Web site's root directory; and

outputting the security vulnerabilities.

30. The process of claim 29 , further comprising scanning at least one of the security vulnerabilities for at least one known exploit in order to identify further security vulnerabilities.

31. The process of claim 29 , further comprising parsing through the linked-to Web pages to identify any further-linked-to Web pages or objects which are included in the linked-to Web pages.

32. The process of claim 31 , further comprising parsing through the first mentioned Web page to identify any hidden Web pages or objects which are included in the hidden Web pages.

33. The process of claim 32 , further comprising parsing through the hidden Web-pages to identify any further-hidden Web pages or objects which are included in the further-hidden Web pages.

34. The process of claim 33 , further comprising:

comparing each hidden Web page and each further-hidden Web page to each linked-to Web page and each further-lipked-to Web page; and

identifying each hidden Web page and each further-hidden Web page that is different from the linked-to Web pages and the further-linked to Web pages.

35. The process of claim 31 , wherein the parsing through the default Web page and the parsing through the linked-to Web pages include performing a keyword search in order to detect at least one point of interest.

36. The process of claim 35 , wherein the at least one point of interest is selected from the group consisting of an administrative Web page and a directory list tag.

37. The process of claim 36 , further comprising automatically passing an authorized username and password to the target Web site, if required to gain access to the target Web site.

38. The process of claim 31 , wherein the scanning the target Web site for at least one known exploit includes checking for at least one common filename.

39. The process of claim 38 , wherein the at least one common filename is selected from the group consisting of “msadcs.dll” and “WS_FTP.LOG.”

40. A system for detecting security vulnerabilities present in a target Web site, comprising:

memory for storing:

a Web page database; and

a security vulnerability database; and

a processor connected to the memory and being configured to:

establish an Internet connection with the target Web site;

retrieve a Web page for the target Web site;

parse through the Web page to identify any linked-to Web pages or objects which are included in the Web page;

identify security vulnerabilities by performing at least one of (i) scanning the target Web site for at least one vulnerability and (ii) apply at least one hack method to the target Web site; and

prioritize the security vulnerabilities.

41. The system of claim 40 wherein the processor identifies security vulnerabilities by both (i) scanning the target Web site for at least one vulnerability and (ii) apply at least one hack method to the target Web site.

42. The system of claim 41 , wherein the processor is configured to apply at least one hack method by attempting to access unauthorized files located outside the target Web site's root directory.

43. The system of claim 42 , wherein the parsing through the first mentioned Web page and the parsing through the linked-to Web pages include performing a keyword search in order to detect at least one point of interest.

44. The system of claim 43 , wherein the at least one point of interest is selected from the group consisting of an administrative Web page and a directory list tag.

45. The system of claim 41 , wherein the processor is configured to apply at least one hack method to the target Web site by automatically passing multiple usernames and passwords to the target Web site if a login Web page is encountered.

46. The system of claim 40 wherein the processor is configured to automatically pass an authorized username and password to the target Web site, if required to gain access to the target Web site.

47. The system of claim 40 , wherein the processor is further configured to parse through the linked-to Web pages to identify any further-linked-to Web pages or objects which are included in the linked-to Web pages.

48. The system of claim 47 , wherein the processor is further configured to scan at least one of the security vulnerabilities for at least one known exploit in order to identify further security vulnerabilities.

49. The system of claim 48 , wherein the processor is further configured to parse through the first mentioned Web page to identify any hidden Web pages or objects which are included in the hidden Web pages.

50. The system of claim 49 , wherein the processor is further configured to parse through the hidden Web-pages to identify any further-hidden Web pages or objects which are included in the further-hidden Web pages.

51. The system of claim 50 , wherein the processor is further configured to:

compare each hidden Web page and each further-hidden Web page to each linked-to Web page and each further-linked-to Web page; and

identify each hidden Web page and each further-hidden Web page that is different from the linked-to Web pages and the further-linked to Web pages.

52. A system for detecting security vulnerabilities present in a target Web site, comprising:

memory for storing:

a Web page database; and

a security vulnerability database; and

a processor connected to the memory and being configured to:

establish an Internet connection with the target Web site;

retrieve a Web page for the target Web site;

parse through the Web page to identify any linked-to Web pages or objects which are included in the Web page, wherein the parsing includes performing a keyword search in order to detect at least one point of interest; and

identify security vulnerabilities by performing at least one of (i) scanning the target Web site for at least one vulnerability and (ii) apply at least one hack method to the target Web site.

53. The system of claim 52 wherein the processor identifies security vulnerabilities by both (i) scanning the target Web site for at least one vulnerability and (ii) apply at least one hack method to the target Web site.

54. The system of claim 52 , wherein the processor is further configured to scan at least one of the security vulnerabilities for at least one know exploit in order to identify further security vulnerabilities.

55. The system of claim 52 , wherein the processor is further configured to parse through the linked-to Web pages to identify any further-linked-to Web pages or objects which are included in the linked-to Web pages.

56. The system of claim 55 , wherein the at least one point of interest is selected from the group consisting of an administrative Web page and a directory list tag.

57. The system of claim 55 , wherein the processor is further configured to parse through the hidden Web-pages to identify any further-hidden Web pages or objects which are included in the further-hidden Web pages.

58. The system of claim 57 , wherein the processor is further configured to:

compare each hidden Web page and each further-hidden Web page to each linked-to Web page and each further-linked-to Web page; and

identify each hidden Web page and each further-hidden Web page that is different from the linked-to Web pages and the further-linked to Web pages.

59. The system of claim 55 , wherein the processor is further configured to automatically pass an authorized username and password to the target Web site, if required to gain access to the target Web site.

60. The system of claim 59 , wherein the processor is configured to apply at least one hack method by attempting to access unauthorized files located outside the target Web site's root directory.

61. The system of claim 55 , wherein the processor is configured to apply at least one hack method by passing invalid data to a data entry field of the target Web site and evaluating the result.

62. The system of claim 61 , wherein the processor is further configured to:

record the invalid data which produces a security vulnerability; and

pass the recorded invalid data to at least one other data entry field of the target Web site.

63. The system of claim 52 , wherein the processor is further configured to parse through the first mentioned Web page to identify any hidden Web pages or objects which are included in the hidden Web pages.

64. A system for detecting security vulnerabilities present in a target Web site, comprising:

memory for storing:

a Web page database;

at least one exploit; and

a security vulnerability database; and

a processor connected to the memory and being configured to:

establish an Internet connection with the target Web site;

retrieve a Web page for the target Web site;

parse through the Web page to identify any linked-to Web pages or objects which are included in the Web page;

scan the target Web site for at least one exploit in order to identify security vulnerabilities; and

apply at least one hack method to the target Web site in order to identify security vulnerabilities, wherein the applying at least one hack method includes attempting to access unauthorized files located outside the target Web site's root directory.

65. The system of claim 64 , wherein the processor is further configured to parse through the linked-to Web pages to identify any further-linked-to Web pages or objects which are included in the linked-to Web pages.

66. The system of claim 65 , wherein the processor is further configured to parse through the first mentioned Web page to identify any hidden Web pages or objects which are included in the hidden Web pages.

67. The system of claim 66 , wherein the processor is further configured to parse through the hidden Web-pages to identify any further-hidden Web pages or objects which are included in the further-hidden Web pages.

68. The system of claim 67 , wherein the processor is further configured to:

compare each hidden Web page and each further-hidden Web page to each linked-to Web page and each further-linked-to Web page; and

identity each hidden Web page and each further-hidden Web page that is different from the linked-to Web pages and the further-linked to Web pages.

69. The system of claim 65 , wherein the parsing through the first mentioned Web page and the parsing through the linked-to Web pages include performing a keyword search in order to detect at least one point of interest.

70. The system of claim 69 , wherein the at least one point of interest is selected from the group consisting of an administration Web page and a directory list tag.

71. The system of claim 64 , wherein the processor is further configured to automatically pass an authorized username and password to the target Web site, if required to gain access to the target Web site.

72. The system of claim 71 , wherein the processor is further configured to scan at least one of the security vulnerabilities for at least one known exploit in order to identify further security vulnerabilities.