ADAPTIVE RATE THROTTLING FOR CLOUD-BASED SERVICE REQUESTS
Some service providers (e.g., Microsoft's Office 365 services) throttle application or service requests an organization, a user, or device can request from a specific application or a suite of applications/services. This throttling can significantly impact performance for, not only the requesting user/device, but also for other tenants associated with that organization. To limit this imposed external throttling, an intelligent internal adaptive throttling or rate limiting may be implemented. In an embodiment, the system monitors throttling imposed by the service provider. Using this information, the system determines an optimal rate of calls that can made for that service to maximize throughput and performance. The system continuously monitors the throttling and in response dynamically modifies the internal target rate.
1 . A method for managing service request throttling from a third-party service, comprising:
obtaining an internal throttling rate target,
wherein the internal throttling rate target represent a maximum number of service calls allowed per time interval to a third-party service provider,
wherein the internal throttling rate target is derived from at least one of an information management policy, defined within source code, or dynamically calculated;
continuously determining a number of service requests to allow in accordance with the internal throttling rate target, wherein the determination is based, at least, on a total number of service requests already made;
periodically:
accessing performance data associated with service requests initiated to the third-party service provider;
determining a current external throttling percent for a preceding period based on the number of service requests throttled by the third-party service provider during the preceding period;
determining that the calculated external throttling percent falls outside a threshold range of a throttling percent target; and
in response to determining that the calculated external throttling percent falls outside a threshold range of a throttling percent target:
calculating a rate correction factor using the throttling percent target and the current external throttling percent, and
adjusting the internal throttling rate target by multiplying the rate correction factor by the current external throttling percent for the preceding period.