Leveraging microservice containers to provide tenant isolation in a multi-tenant API gateway
A system can host APIs for a plurality of different tenants and receive requests from many different client devices. As requests are received, an associated tenant can be identified, and a router can determine if a container instance is available to service the request. A container instance may be an empty container instance including an internal endpoint, a Web server, and a runtime environment. An empty container instance can be unassociated with a particular tenant. To associate a container instance with a tenant, a data store, such as a key-value data store can retrieve configuration files that turn the agnostic container instance into a container instance that is associated with particular tenant and includes configuration code to perform the requisite API functions. The pool of empty and populated containers can be managed efficiently.
1. A method of managing runtime states and service configurations for containers in a multi-tenant environment, the method comprising:
receiving an indication that a request has been received for a service provided in the multi-tenant environment;
identifying a configuration that implements the service, wherein the configuration is stored in a data store;
sending the configuration to a container in the multi-tenant environment to service the request;
receiving a runtime state from the container; and
storing the runtime state in the data store, wherein the configuration is flushed from the container.
2. The method of claim 1 , wherein the configuration is provided by a tenant of the multi-tenant environment prior to runtime.
3. The method of claim 1 , wherein the data store comprises a key-value data store.
4. The method of claim 3 , wherein the key-value data store comprises a distributed key-value data store.
5. The method of claim 3 , wherein an identity of a tenant associated with the service comprises a key in the key-value data store, and wherein the configuration and runtime state comprise a value in the key-value data store.
6. The method of claim 1 , wherein the data store also stores a registry of containers that are available in the multi-tenant environment.
7. The method of claim 6 , wherein the data store updates one or more routers in the multi-tenant environment when new containers become available in the multi-tenant environment based on the registry of containers.
8. The method of claim 1 , further comprising:
receiving an indication that a second request has been received for the service after the configuration is flushed from the container;
sending the configuration to a second container in the multi-tenant environment to service the second request;
sending the runtime state to the second container;
receiving an updated runtime state from the second container; and
storing the updated runtime state in the data store.
9. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, causes the one or more processors to perform operations comprising:
receiving an indication that a request has been received for a service provided in a multi-tenant environment;
identifying a configuration that implements the service, wherein the configuration is stored in a data store;
sending the configuration to a container in the multi-tenant environment to service the request;
receiving a runtime state from the container; and
storing the runtime state in the data store, wherein the configuration is flushed from the container.
10. The non-transitory computer-readable medium of claim 9 , wherein the operations further comprise:
assigning a plurality of containers to a first tenant in the multi-tenant environment;
identifying one or more containers in the plurality of containers that are assigned to the first tenant but that are not being used by the first tenant;
flushing contents of the one or more containers; and
reassigning the one or more containers to a second tenant in the multi-tenant environment.
11. The non-transitory computer-readable medium of claim 10 , wherein after flushing the contents of the one or more containers, the one or more containers are not assigned to any tenant for a first time interval before being reassigned to the second tenant.
12. The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:
identifying second one or more containers in the plurality of containers that are assigned to the first tenant but that are not being used by the first tenant;
determining that no other tenants need the second one or more containers; and
removing the second one or more containers from the multi-tenant environment.
13. The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:
determining that the first tenant is receiving more requests than can be serviced by the plurality of containers.
14. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise:
instantiating a new plurality of containers; and
assigning the new plurality of containers to the first tenant.
15. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise:
assigning containers that were previously assigned to another tenant to the first tenant.
16. The non-transitory computer-readable medium of claim 10 , wherein a gateway of the multi-tenant environment reassigns the one or more containers to the second tenant in the multi-tenant environment.
17. A system comprising:
one or more processors; and
one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving an indication that a request has been received for a service provided in a multi-tenant environment;
identifying a configuration that implements the service, wherein the configuration is stored in a data store;
sending the configuration to a container in the multi-tenant environment to service the request;
receiving a runtime state from the container; and
storing the runtime state in the data store, wherein the configuration is flushed from the container.
18. The system of claim 17 , wherein the operations further comprise:
receiving a first request for a first service provided by a first tenant;
selecting an empty container in the multi-tenant environment;
loading a first configuration that implements the first service into the container;
servicing the first request from the container;
receiving a second request for a second service provided by a second tenant;
flushing the first configuration from the container, wherein after flushing the first configuration from the container, the container comprises a runtime process with an embedded server and an internal endpoint; and
servicing the second request from the container.
19. The system of claim 18 , wherein the container is one of a plurality of containers in the multi-tenant environment that are instantiated to service requests from client devices.
20. The system of claim 18 , wherein the first configuration comprises a size of a heap in memory that can be used by the first service.