Selective website vulnerability and infection testing
In embodiments of the present invention improved capabilities are described for selective website vulnerability and infection testing and intelligently paced rigorous direct website testing. By providing robust website content integrity checking while only lightly loading the website hosting server, visitor bandwidth availability is maintained through selective testing and intelligently paced external website exercising. A modular pod-based computing architecture of interconnected severs configured with a sharded database facilitates selective website testing and intelligent direct website test pacing while providing scalability to support large numbers of website testing subscribers.
1. A method for moderating bandwidth consumption of a hosting system for a set of commonly hosted websites on which vulnerability and infection testing is to be performed, the method comprising:
determining, with a computer remotely connected to the set of commonly hosted websites over a network, an expected bandwidth consumption by users due to non-testing access to at least a portion of the set of commonly hosted websites of the hosting system at a time when the vulnerability and infection testing of the commonly hosted websites is to commence;
comparing, with the computer remotely connected to the set of commonly hosted websites, the determined expected bandwidth consumption by users of the hosting system to a predetermined threshold value;
selecting, with the computer remotely connected to the set of commonly hosted websites, at least one of lower bandwidth consuming procedures and higher bandwidth consuming procedures of the vulnerability and infection testing to be performed at the time when infection testing is to commence, wherein the higher bandwidth consuming procedures are selected if the determined expected bandwidth consumption by users of the hosting system is lower than the predetermined threshold value;
selecting, by the computer remotely connected to the set of commonly hosted websites, a website from the set of commonly hosted websites; and
scanning, by the computer remotely connected to the set of commonly hosted websites, the content of the selected website at a pace determined by the selected higher or lower bandwidth consuming procedures.
2. The method of claim 1 , further comprising determining the pacing of the selected procedures based upon the determined bandwidth consumption by users of the hosting system.
3. The method of claim 1 , wherein the vulnerability and infection testing is performed by a plurality of website security services.
4. The method of claim 1 , wherein the vulnerability and infection testing for at least one of the commonly hosted websites includes accessing one or more pages of the at least one of the commonly hosted websites.
5. The method of claim 4 , further comprising capturing links from one or more pages of the at least one of the commonly hosted websites and comparing the captured links to a link analysis dataset comprising at least one of whitelisted links and black-listed links.
6. The method of claim 1 , further comprising a step of selecting for vulnerability and infection testing at least one page of one of the commonly hosted websites based on the at least one page having been found to have contained a vulnerability or an infection during a prior vulnerability and infection test.
7. The method of claim 1 , further comprising a step of selecting for vulnerability and infection testing a first page of one of the commonly hosted websites before a second page of that website wherein the first page contains a link that was identified as infected by previous vulnerability and infection testing and the second page does not contain a link that was not previously identified as infected by previous vulnerability and infection testing.
8. The method of claim 1 , wherein the vulnerability and infection testing includes scanning a subset of link string page generation variables.
9. The method of claim 1 , further comprising a step of comparing a Uniform Resource Locator (URL) length of a first page of one of the commonly hosted websites with a URL length of a second page of that commonly hosted website, and a step of performing the vulnerability and infection testing on whichever of the first and second pages has the shortest length before performing the vulnerability and infection testing of the other of the first and second pages.
10. The method of claim 1 , wherein the selecting step includes a determination of which pages of the commonly hosted websites to test.
11. The method of claim 1 , wherein the first set of lower consuming bandwidth procedures is selected over the second set of higher consuming bandwidth procedures if the determined loading level is greater than a predetermined threshold.
12. The method of claim 1 , further including performing the selected procedures prior to a subsequent testing invocation.