Proactive management of service connections
Techniques for a Predictive Connection Manager Service (PCMS) to predict when client applications will send service requests to backend services, and proactively establishes connections, caches data, or takes other actions, to reduce latencies between receipt of and response to these service requests. The PCMS analyzes historical usage data for the client applications to identify usage patterns, and uses those usage patterns to proactively scale resources to handle service requests. The PCMS can be implemented as a pass-through proxy for client applications to reduce frictions for managing how users interact with backend services. For instance, the PCMS can install client-side drivers such that updates or patches for the drivers need only be installed on the PCMS rather than on each client device. Further, the PCMS provides interfaces through which users can develop custom drivers for backend services, and also manages software drivers for different service provider networks, thus offering multi-provider connectors.
1 . A computer-implemented performed at least partly by a client device, comprising:
running a client application on the client device, wherein the client application establishes connections between the client device and a backend service of a service provider network;
storing, on the client device, a predictive model that models a usage pattern according to which the client application established historical connections with the backend service;
determining, by a connection manager running on the client device and using the predictive model, a time at which the client application is likely to request a connection to the backend service;
causing the client device to establish the connection with the backend service prior to the time; and
providing the client application with access to use the connection with the backend service at least by the time.
2 . The method of claim 1 , wherein the connection manager is included in an application stack of the client application such that the connection manager is executed as part of the client application being run.
3 . The method of claim 1 , further comprising:
running a container on the client device;
running the connection manager in the container; and
storing the predictive model in the container.
4 . The method of claim 1 , further comprising:
determining, using the predictive model, a number of connections to the backend service that the client application is likely to request at the time;
establishing, by the connection manager and prior to the time, the number of connections with the backend service; and
providing the client application with access to use the number of connections with the backend service at least by the time.
5 . The method of claim 4 , further comprising:
determining, by the connection manager, a rate limit indicating a threshold number of connections that the client application is permitted to establish during a period of time;
identifying a number of connection requests initiated by the client application that violates the rate limit; and
preventing, by the connection manager, the client application from establishing a number of connections on behalf of the client application that exceeds the rate limit.
6 . The method of claim 1 , further comprising:
determining, by the connection manager, particular data that the client application is likely to request from the backend service at the time;
obtaining, by the connection manager, the particular data from the backend service prior to the time;
caching the particular data in a data cache of the client device prior to the time; and
retrieving, by the connection manager, the particular data from the data cache in response to the client application requesting the particular data.
7 . The method of claim 1 , wherein the time is a first time, further comprising:
establishing a first number of connections between the client device and the backend service for the client application;
predicting, by the connection manager, a second time at which the client application is to reduce usage of the first number of connections to usage of a second number of connections; and
reducing the first number of connections to the second number of connections based at least in part on the second time.
8 . A client device comprising:
one or more network interfaces;
one or more processors; and
one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
executing an application that establishes connections between the client device and a backend service of a service provider network;
storing a predictive model that models a usage pattern according to which the client application established historical connections with the backend service;
determining, by a connection manager running on the client device and using the predictive model, (i) a time at which the client application is likely to request connections to the backend service, and (ii) a number of connections to the backend service that the client application is likely to request at the time;
establishing, using the one or more network interfaces, the number of connections with the backend service prior to the time; and
providing the client application with access to use the number of connections with the backend service at least by the time.
9 . The client device of claim 8 , further comprising memory, the operations further comprising:
storing an application stack of the client application in the memory; and
storing code of the connection manager in a layer of the application stack;
wherein the connection manager is executed as part of the client application being executed.
10 . The client device of claim 8 , the operations further comprising:
executing a container;
executing the connection manager in the container; and
storing the predictive model in the container.
11 . The client device of claim 8 , the operations further comprising:
determining, by the connection manager, a rate limit indicating a threshold number of connections that the client application is permitted to establish during a period of time;
identifying a number of connection requests initiated by the client application that violates the rate limit; and
preventing, by the connection manager, the client application from establishing a number of connections on behalf of the client application that exceeds the rate limit.
12 . The client device of claim 8 , the operations further comprising:
determining, by the connection manager, particular data that the client application is likely to request from the backend service at the time;
obtaining, by the connection manager, the particular data from the backend service prior to the time;
caching the particular data in a data cache of the client device prior to the time; and
retrieving, by the connection manager, the particular data from the data cache in response to the client application requesting the particular data.
13 . The client device of claim 8 , wherein the time is a first time, the operations further comprising:
establishing a first number of connections between the client device and the backend service for the client application;
predicting, by the connection manager, a second time at which the client application is to reduce usage of the first number of connections to usage of a second number of connections; and
reducing the first number of connections to the second number of connections based at least in part on the second time.
14 . A method comprising:
receiving, at a connection-manager service, historical data that represents historical connections previously established between a client application of a client device and a backend service of a service provider network;
determining, by the connection-manager service and using the historical data, a time at which the client application is likely to request particular data from the backend service;
caching a set of permissions associated with the client application;
prior to obtaining the particular data, determining that the client application is permitted to access the particular data based at least in part on the set of permissions;
obtaining, at the connection-manager service and prior to the time, the particular data from the backend service;
caching the particular data in a data cache of the connection-manager service; and
providing the client application with access to the particular data at least by the time.
15 . The method of claim 14 , further comprising:
determining, using the historical data, that the client application is likely to request a connection to the backend service at the time;
establishing, prior to the time, the connection between the connection-manager service and the backend service; and
providing the client application with access to use the connection with the backend service at least by the time.
16 . The method of claim 14 , further comprising:
determining, using the historical data, a number of connections to the backend service that the client application is likely to request at the time;
establishing, by the connection-manager service and prior to the time, the number of connections with the backend service; and
providing the client application with access to use the number of connections with the backend service at least by the time.
17 . The method of claim 14 , further comprising:
establishing, by the connection-manager service, a set of connections with the backend service on behalf of the client application;
monitoring, by the connection-manager service, the set of connections established on behalf of the client application;
determining a metric associated with the set of connections; and
providing a user associated with the client application with access to the metric.
18 . The method of claim 14 , further comprising establishing a set of connections with the backend service on behalf of the client application, wherein:
the connection-manager service comprises a cluster of containers running in a data center of the service provider network, the connection-manager service serving as a proxy between the client device and the backend service;
individual containers of the cluster of containers establish individual connections of the set of connections with the backend service; and
establishing the set of connections with the backend service includes scaling up a respective number of containers in the cluster of containers to establish the set of connections.
19 . The method of claim 14 , further comprising:
determining a rate limit indicating a threshold number of connections that the client application is permitted to establish during a period of time;
receiving, from the client application, a number of connection requests that violates the rate limit; and
refraining from establishing a number of connections on behalf of the client application that exceeds the rate limit.
20 . A method comprising:
receiving, at a connection-manager service, historical data that represents historical connections previously established between a client application of a client device and a backend service of a service provider network;
determining, by the connection-manager service and using the historical data, a time at which the client application is likely to request particular data from the backend service;
obtaining, at the connection-manager service and prior to the time, the particular data from the backend service;
caching the particular data in a data cache of the connection-manager service;
providing the client application with access to the particular data at least by the time;
determining a rate limit indicating a threshold number of connections that the client application is permitted to establish during a period of time;
receiving, from the client application, a number of connection requests that violates the rate limit; and
refraining from establishing a number of connections on behalf of the client application that exceeds the rate limit.