Adaptive rate limiter based on transactional heuristics and artificial intelligence
A system, method, and device for adaptively limiting web requests is disclosed. The method includes (i) receiving a request at a proxy for an authentication service, (ii) determining whether the request is trusted based at least in part on a classification obtained from a classifier, and (iii) handling the request according to a determination of whether the request is trusted.
1 . A system for adaptively limiting web requests, comprising:
one or more processors configured to:
receive a request at a proxy that mediates traffic between client systems and an authentication service;
determine whether the request is trusted based at least in part on a classification obtained from a classifier, wherein the classifier generates the classification based at least in part on whether credentials are returned by the requesting entity in response to an authentication challenge issued by the authentication service; and
handle the request according to a determination of whether the request is trusted, wherein handling the request includes adaptively adjusting a rate at which authentication requests for the authentication service are processed by (i) rate-limiting requests classified as untrusted and (ii) prioritizing processing of requests classified as trusted, the adjusting being performed based at least in part on the classification that reflects the credential-return observation; and
a memory coupled to the one or more processors and configured to provide the one or more processors with instructions.
2 . The system of claim 1 , wherein handling the request comprises processing the request in response to determining that the request is trusted.
3 . The system of claim 1 , wherein the classifier is a machine learning model.
4 . The system of claim 1 , wherein the classifier is trained based at least in part on XGBoost machine learning process.
5 . The system of claim 1 , wherein handling the request comprises blocking the request in response to determining that the request is untrusted.
6 . The system of claim 1 , wherein the classifier provides a binary classification of the request as either a trusted request or an invalid request.
7 . The system of claim 1 , wherein the request is deemed trusted in response to the classification predicting that the request is legitimate.
8 . The system of claim 1 , wherein the request is deemed untrusted in response to the classification predicting that the request is part of a distributed denial of service attack.
9 . The system of claim 1 , wherein if the request is deemed untrusted handling the request comprises returning an indication that the request will not be processed.
10 . The system of claim 1 , wherein the classifier generates a predicted classification based at least in part on one or more attributes associated with the request.
11 . The system of claim 1 , wherein handling the request according to a determination of whether the request is trusted comprises:
obtain an IP address associated with the proxy via which the request is sent; and
configure a security entity to block requests associated with the IP address for a predefined period of time, wherein the predefined period of time is based on a security policy.
12 . The system of claim 11 , wherein the classification for the request is reset after the predefined period of time.
13 . The system of claim 11 , wherein requests from the IP address are re-classified after predefined period of time.
14 . The system of claim 1 , wherein handling the request according to the determination of whether the request is trusted comprises:
rate-limiting requests from the proxy based at least in part on determining that the classification for the request is indicative of the request being untrusted, wherein the rate-limiting is dynamic based on a probability score from the classification.
15 . The system of claim 1 , wherein the request is classified based at least in part on the request and a response to an authentication challenge.
16 . The system of claim 15 , wherein the response is a response to the authentication service for authenticating a user or device associated with the request.
17 . The system of claim 1 , wherein the predefined period of time for blocking requests associated with the IP address is dynamically determined based at least in part on a severity of the untrusted classification or a frequency of untrusted requests from the IP address.
18 . The system of claim 1 , wherein after the classification for the request is reset, the system is configured to forward a subsequent request from the same IP address to the authentication service to determine whether the subsequent request is legitimate or illegitimate based on whether credentials are returned in response to an authentication challenge for the subsequent request.
19 . The system of claim 1 , wherein the system is configured to:
after the predefined period of time for blocking requests associated with the IP address has expired, reset the classification associated with the IP address; and
forward a subsequent request from the same IP address to the authentication service to determine whether the subsequent request is legitimate or illegitimate based on whether credentials are returned by the requesting entity in response to an authentication challenge issued by the authentication service for the subsequent request.
20 . The system of claim 1 , wherein:
the adaptive adjustment of the rate at which authentication requests are processed is performed dynamically based on a probability score output by the classifier, the probability score reflecting the likelihood that credentials will be returned in response to an authentication challenge; and
the system rate-limits authentication requests having a probability score below a predetermined threshold while prioritizing and forwarding authentication requests having a probability score above the predetermined threshold.
21 . A method for adaptively limiting web requests, comprising:
receiving a request at a proxy that mediates traffic between client systems and an authentication service;
determining whether the request is trusted based at least in part on a classification obtained from a classifier, wherein the classifier generates the classification based at least in part on whether credentials are returned by the requesting entity in response to an authentication challenge issued by the authentication service; and
handling the request according to a determination of whether the request is trusted, wherein handling the request includes adaptively adjusting a rate at which authentication requests for the authentication service are processed by (i) rate-limiting requests classified as untrusted and (ii) prioritizing processing of requests classified as trusted, the adjusting being performed based at least in part on the classification that reflects the credential-return observation.
22 . A computer program product embodied in a non-transitory computer readable medium for adaptively limiting web requests, and the computer program product comprising computer instructions for:
receiving a request at a proxy that mediates traffic between client systems and an authentication service;
determining whether the request is trusted based at least in part on a classification obtained from a classifier, wherein the classifier generates the classification based at least in part on whether credentials are returned by the requesting entity in response to an authentication challenge issued by the authentication service; and
handling the request according to a determination of whether the request is trusted, wherein handling the request includes adaptively adjusting a rate at which authentication requests for the authentication service are processed by (i) rate-limiting requests classified as untrusted and (ii) prioritizing processing of requests classified as trusted, the adjusting being performed based at least in part on the classification that reflects the credential-return observation.
23 . A system for training a classifier for adaptively limiting web requests, comprising:
one or more processors configured to:
obtain a set of requests at a proxy for an authentication service;
train a classifier based at least in part on the set of requests, wherein the classifier is configured to identify legitimate requests to the proxy for the authentication service; and
deploy the classifier in connection with adaptively limiting requests to the proxy for the authentication service based on one or more classifications predicted by the classifier;
wherein:
the classifier generates a particular classification based at least in part on whether credentials are returned in response to an authentication challenge for a particular request; and
the adaptively limiting comprises adjusting a rate at which authentication requests are processed based at least in part on the one or more classifications predicted by the classifier; and
a memory coupled to the one or more processors and configured to provide the one or more processors with instructions.
24 . The system of claim 23 , wherein the set of requests are obtained based on a log of requests at the authentication service.
25 . The system of claim 23 , wherein the set of requests correspond to past authentication requests.