Regulation methods for proxy services
Systems and methods to manage and regulate the requests of multiple proxy clients are disclosed. In one aspect, the system and methods disclosed herein aids in configuring proxy server(s) with a rate-limit functionality. Configuration of the rate-limit functionality may be realized by, but not limited to, installing configuration file(s) and/or software application(s) on the proxy server(s). The configuration provides information about the list of restricted and unrestricted domains and their respective request limit specification in a given time frame. Therefore, each time before a proxy server forwards the clients' requests to a target domain, the proxy server checks and ensures that the request count to the particular target domain is well within the limit specified in the request limit specification. Thus, the embodiments described herein aid in preventing the IP addresses of proxy service providers from being blocked or denied from the target websites.
1. A system to regulate and upgrade proxy servers comprising:
a proxy regulator, operable to install a configuration file on a proxy server, to store configuration setting and metadata information about the proxy server in a proxy database, and to select proxy server information from the proxy database;
a gateway, operable to communicate with a client, to receive a proxy server request from a client device about the proxy server, and to forward the proxy server request to the proxy regulator;
the proxy regulator, operable to send proxy server information from the proxy database to the client device through the gateway; and
the proxy server with the installed configuration file, upon receiving a client request from the client device, operable to check a target domain indicated in the client request to determine whether the target domain is in a list of restricted domains or a list of unrestricted domains;
if the target domain is in the list of unrestricted domains, sending the client request to the target domain;
if the target domain is in the list of restricted domains, checking whether a request count for the target domain has reached a limit in a time frame; and:
if the request count for the target domain is below the limit, the proxy server forwards the client request to the target domain;
if the request count for the target domain is over the limit, the proxy server sends an error message to the client device;
wherein the configuration file enables the proxy server to manage incoming requests and outgoing requests.
2. The system of claim 1 , wherein the configuration file includes one, or a combination, of:
the list of restricted domains and related sub-domains to which a rate-limiting functionality is applicable;
the list of unrestricted domains and related sub-domains to which the rate-limiting functionality is not applicable;
request limit specifications of each restricted domain for an IP address unique to the proxy server;
request limit specifications of each restricted domain for a set of multiple unique IP addresses unique to the proxy server.
3. The system of claim 1 , wherein the limit indicates a number of requests that can be submitted or forwarded to each of the list of restricted domains in the time frame.
4. The system of claim 1 , wherein the client request comprises at least one, or a combination, of: Uniform Resource Locator (URL) of the target domain, relevant queries seeking data, authentication credentials.
5. The system of claim 4 , wherein the authentication credentials can comprise at least one of or a combination of: usernames/IDs, passwords, personal identification numbers (PINs), serial numbers, or one-time passwords (OTPs).
6. The system of claim 1 , wherein the proxy server receives the client request from the client device.
7. The system of claim 1 , wherein the proxy server, upon receiving the client request executes an authentication and authorization process.
8. The system of claim 1 , wherein the proxy server checks the target domain indicated in the client request to determine whether the target domain is in the list of restricted domains or in the list of unrestricted domains only when an authentication and authorization process is successful.
9. The system of claim 1 , wherein the proxy regulator finds the proxy server information from the proxy database.
10. A non-transitory computer readable medium comprising instructions that, when executed by one or more processors in a network, direct the one or more processors to:
install, by a proxy regulator, a configuration file on a proxy server:
store, by the proxy regulator, configuration setting and metadata information about the proxy server in a proxy database;
receive, by a gateway, a proxy server request about the proxy server from a client device;
forward, by the gateway, the proxy server request to the proxy regulator,
send, by the proxy regulator, proxy server information from the proxy database to the client device through the gateway;
upon receiving a client request from the client device, check, by the proxy server, a target domain indicated in the client request to determine whether the target domain is in a list of restricted domains or in a list of unrestricted domains;
wherein the check involves:
if the target domain is in the list of unrestricted domains, sending the client request to the target domain;
if the target domain is in the list of restricted domains, checking whether a request count for the target domain has reached a limit in a time frame; and:
if the request count for the target domain is below the limit, the proxy server forwards the client request to the target domain;
if the request count for the target domain is over the limit, the proxy server sends an error message to the client device;
wherein the configuration file enables the proxy server to manage incoming requests and outgoing requests.
11. The non-transitory computer readable medium of claim 10 , wherein the configuration file includes one, or a combination, of:
the list of restricted domains and related sub-domains to which a rate-limiting functionality is applicable;
the list of unrestricted domains and related sub-domains to which the rate-limiting functionality is not applicable;
request limit specifications of each restricted domain for an internet protocol (IP) address unique to the proxy server;
request limit specifications of each restricted domain for a set of multiple unique IP addresses unique to the proxy server.
12. The non-transitory computer readable medium of claim 10 , wherein the limit indicates a number of requests that can be submitted or forwarded to each of the list of restricted domains in the time frame.
13. The non-transitory computer readable medium of claim 10 , wherein the client request comprises at least one, or a combination, of: Uniform Resource Locator (URL) of the target domain, relevant queries seeking data, authentication credentials.
14. The non-transitory computer readable medium of claim 10 , wherein the authentication credentials can comprise at least one of or a combination of: usernames/IDs, passwords, PINs, serial numbers, or OTPs.
15. The non-transitory computer readable medium of claim 10 , wherein the client request is received from a client device.
16. The non-transitory computer readable medium of claim 10 , wherein the proxy server receives the client request from the client device.
17. The non-transitory computer readable medium of claim 10 , wherein the proxy server, upon receiving the client request executes an authentication and authorization process.
18. The non-transitory computer readable medium of claim 10 , wherein the proxy server checks the target domain indicated in the client request to determine whether the target domain is in the list of restricted domains or in the list of unrestricted domains only when an authentication and authorization process is successful.