Methods and systems for implementing a regionally contiguous proxy service
System and method for efficiently implementing scalable, highly efficient decentralized proxy services through proxy infrastructures situated in different geo-locations. In one aspect, the systems and methods enable users from any geographical location to send requests to the geographically closest proxy infrastructure. One exemplary method described allows proxy infrastructures to gather, classify, and store metadata of exit nodes in its internal database. In another aspect, systems and methods described herein enable proxy infrastructures to select metadata of exit nodes from its internal database and forward requests from a user device to respective proxy servers or proxy supernodes to which the selected exit nodes are connected.
1. A method for data extraction and gathering from a web server using a nearest available proxy server comprising:
receiving, by a proxy gateway at a proxy infrastructure geographically closest to a user device, a user request;
forwarding, by the proxy gateway at the proxy infrastructure, the user request to a proxy messenger;
checking, by the proxy messenger at the proxy infrastructure, the user request for geolocation of an exit node;
requesting, by the proxy messenger at the proxy infrastructure, from a repository unit, metadata of the exit node to implement the user request;
selecting, by the repository unit at the proxy infrastructure from a pool database, the exit node to implement the user request;
sending, by the proxy messenger at the proxy infrastructure, the user request and an Internet Protocol (IP) address of the exit node selected to a proxy supernode;
instructing, by the proxy supernode, the exit node selected to establish a connection with a target;
sending, by the proxy supernode, the user request to the exit node selected;
forwarding, by the exit node selected, the user request to the target.
2. The method of claim 1 , wherein content for the user request is returned to a User device from the target by:
receiving, by the exit node selected, the response for the user request from the target;
forwarding, by the exit node selected, the response received from the target to the proxy supernode;
forwarding, by the proxy supernode, the response received from the target to the proxy messenger;
forwarding, by the proxy messenger, the response received from the target to the proxy gateway;
forwarding, by the proxy gateway, the response received from the target to the user device.
3. The method of claim 1 , wherein the repository unit at the proxy infrastructure gathers from a central management unit the metadata of multiple exit nodes irrespective of their locations.
4. The method of claim 1 , wherein exit nodes in the pool database are classified into categories based on exit node attributes, comprising: geo-location, ability to reach specific targets, or latency/response time.
5. The method of claim 1 , wherein the proxy supernode maintains connection with multiple exit nodes in geographical proximity.
6. The method of claim 5 wherein the exit node connects to the proxy supernode by making a Domain Name System (DNS) query to a regional DNS server.
7. The method of claim 1 wherein, before accepting the user request, the proxy messenger at the proxy infrastructure, performs user validation actions including checking data about a user in a user database.
8. The method of claim 7 wherein the request from the user device contains authentication credentials necessary for user validation.
9. The method of claim 1 wherein the user device connects to the geographically closest proxy infrastructure by requesting from the DNS provider an Internet Protocol (IP) address of the geographically closest proxy infrastructure.
10. The method of claim 1 wherein the user request contains user-defined preferences comprising: geographical location of the exit node, ability to reach specific target, operating system platform or system configuration settings.
11. A method for data extraction and gathering from a web server using a nearest available proxy server comprising:
receiving, by a proxy gateway at a proxy infrastructure geographically closest to a user device, a user request;
forwarding, by the proxy gateway at the proxy infrastructure, the user request to a proxy messenger;
checking, by the proxy messenger at the proxy infrastructure, the user request for geolocation of an exit node;
requesting, by the proxy messenger at the proxy infrastructure, from a repository unit, metadata of the exit node to implement the user request;
selecting, by the repository unit at the proxy infrastructure from a pool database, the exit node to implement the user request;
sending, by the proxy messenger at the proxy infrastructure, the user request and an Internet Protocol (IP) address of the exit node selected to a proxy supernode;
instructing, by the proxy supernode, the exit node selected to establish a connection with a target;
sending, by the proxy supernode, the user request to the exit node selected;
forwarding, by the exit node selected, the user request to the target;
wherein the exit node selected to implement the user request is in the closest geographical proximity to the proxy infrastructure, wherein the user request lacks geolocation preference, and wherein the repository unit identifies, selects and submits to the proxy messenger the metadata of the exit node according to the proxy messenger requirements.
12. The method of claim 1 wherein the exit node selected to implement the user request is in the closest geographical proximity to the target, and wherein the repository unit identifies and selects the metadata of the exit node situated closest to the target, wherein the geolocation of the target is deduced from the parameters of the user request.
13. The method of claim 1 wherein the user request indicates the specific exit node geolocation, and wherein the repository unit identifies, selects and submits to the proxy messenger the metadata of the exit node according to the proxy messenger requirements that satisfies user-defined preferences.
14. The method of claim 1 wherein metadata of the exit node sent by the repository unit contains IP address information of the exit node selected and the proxy supernode to which the exit node selected is connected.
15. The method of claim 1 wherein the proxy supernode reports gathered information regarding the exit node to a central management unit on a regular basis.
16. The method of claim 1 , wherein the exit node is connected to the proxy supernode geographically closest to the exit node.
17. A system for data extraction and gathering from a web server using a nearest available proxy server comprising:
a proxy gateway, a proxy messenger, and a repository unit, the proxy gateway being within a proxy infrastructure geographically closest to a user device operable to receive a user request;
the proxy messenger operable to check the user request for geolocation of exit nodes, to request from the repository unit, metadata of the exit nodes to implement the user request;
the repository unit operable to select an exit node of the exit nodes to implement the user request;
the proxy messenger operable to send the user request and an Internet Protocol (IP) address of the exit node selected to a proxy supernode;
the proxy supernode operable to instruct the exit node selected to establish a connection with a target and to send the user request to the exit node selected;
the exit node selected operable to forward the user request to the target.
18. The system of claim 17 , wherein content for the user request is returned to a user device from the target by:
receiving, by the exit node selected, the response for the user request from the target;
forwarding, by the exit node selected, the response received from the target to the proxy supernode;
forwarding, by the proxy supernode, the response received from the target to the proxy messenger;
forwarding, by the proxy messenger, the response received from the target to the proxy gateway;
forwarding, by the proxy gateway, the response received from the target to the user device.
19. The system of claim 17 , wherein the repository unit at the proxy infrastructure gathers from a central management unit the metadata of multiple exit nodes irrespective of their locations.
20. The system of claim 17 , wherein the exit nodes in the pool database are classified into categories based on exit node attributes, comprising: geo-location, ability to reach specific targets, or latency/response time.