System and method for enabling a web site robot trap
View Patent ↗The invention allows a system to prevent robots from browsing a Web site beyond a welcome page. When an initial request from an undefined originator is received Web site responds to it with a welcome page including at least one trap. Then, on receiving further requests from the undefined originator Web site can check whether a trap is used or not. If not used the undefined originator is assumed to be a human being and is authorized to go on. If a trap is however used the undefined originator is assumed to be a robot in which case site access is further denied. The invention prevents Web site contents from being investigated by robots while not requiring users to have to log on.
1. A method for preventing robots from browsing a Web site beyond a welcome page, said method in said Web site comprising the steps of:
on receiving an initial request from an undefined originator to establish a session:
responding to said initial request with a said welcome page including at least one trap;
dynamically monitoring action by the undefined originator on said at least one trap during said session;
on receiving further requests from said undefined originator during said session:
checking whether one of said at least one trap is used or not;
if not used:
assuming that said undefined originator is a human being and keep processing said further requests; and
if used:
assuming that said undefined originator is a robot and stopping processing of said further requests.
2. The method according to claim 1 further including the steps of:
logging a source IP address of said undefined originator and starting a timer.
3. The method according to claim 2 further including, whenever receiving a new request, the steps of:
checking whether a source IP address of said new request is matching said logged source IP address; and
if matching:
checking whether said timer has expired or not; and
if expired:
resetting said logged IP address; and
proceeding normally with said new request;
if not expired:
performing one of dropping and redirecting said TCP connection; and
if not matching:
proceeding normally with said new request; and
repeating said steps with every new received request.
4. The method according to claim 1 wherein said step of stopping processing said further requests includes performing one of dropping and redirecting a TCP connection established with said undefined originator.
5. The method according to claim 1 wherein said trap is an HTML anchor invisible when displayed by a Web browser.
6. The method according to claim 5 wherein said HTML anchor includes an invisible image.
7. The method according to claim 1 wherein every page out of said Web site includes at least one trap.
8. A system for a Web site to implement at least one trap in a Web page for preventing robots from browsing a Web site beyond a welcome page, comprising:
a processor comprising a response component, a trap checking component and processing component;
response component for, on receiving an initial request from an undefined originator to establish a session, responding to said initial request with a welcome page including at least one trap;
trap checking component for, dynamically monitoring action by the undefined originator on said at least one trap during said session, and on receiving further requests from said undefined originator in the same session, checking whether one of said at least one trap is used or not;
processing component for, if said at least one trap is not used, processing said further requests; and if said at least one trap is used, stopping processing of said further requests.
9. The system of claim 8 further comprising logging means for logging a source IP address of said undefined originator and starting a timer.
10. The system of claim 9 wherein said stopping processing by said processing component comprises:
performing one of dropping and redirecting a TCP connection established with said undefined originator.
11. The system according to claim 8 further including means for checking whether a source IP address of said new request is matching said logged source IP address; and
if matching:
checking whether said timer has expired or not; and
if expired:
resetting said logged IP address; and
proceeding normally with said new request;
if not expired:
performing one of dropping and redirecting said TCP connection; and
if not matching:
proceeding normally with said new request; and
repeating said steps with every new received request.
12. A computer like readable medium comprising instructions for carrying out a method for preventing robots from browsing a Web site beyond a welcome page, said method in said Web site comprising the steps of:
on receiving an initial request from an undefined originator to establish a session:
responding to said initial request with a welcome page including at least one trap;
monitoring action by the undefined originator on said at least one trap during said session;
on receiving further requests from said undefined originator in the same session:
checking whether one of said at least one trap is used or not;
if not used:
assuming that said undefined originator is a human being and keep processing said further requests; and
if used:
assuming that said undefined originator is a robot and stopping processing of said further requests.