Remotely managing execution of containerized applications across user-managed and server-managed data centers
Generally described, the present application relates to providing a container orchestration service that can enable and manage execution of containerized applications on user-owned infrastructure and cloud-provided compute capacity. In some embodiments, a request to execute a task may indicate the type of compute capacity (e.g., internal/external, computing resource amount, etc.) to be used to execute the task. For example, if the task indicates that internal compute capacity is to be used, compute capacity hosted the cloud provider network can be identified and used to execute the task. Alternatively, if the task indicates that external compute capacity is to be used, instructions for executing the task can be generated and sent to the user-owned infrastructure, and the task can be executed using compute capacity provided within the user-owned infrastructure, which is external to the cloud provider network implementing the container orchestration service.
1 . A cloud provider system comprising:
a first set of physical machines hosting internal compute capacity comprising a plurality of compute instances usable to execute tasks;
a set of capacity providers associated with a cluster of compute capacity and configured to provide compute capacity to the cluster for executing tasks in the cluster, wherein the set of capacity providers includes (i) a first capacity provider for providing internal compute capacity hosted by the first set of physical machines of the cloud provider system, and (ii) a second capacity provider for providing external compute capacity hosted by a second set of physical machines external to the cloud provider system; and
a container service comprising computer hardware, wherein the container service is configured to at least:
receive, from a user computing device, one or more threshold values, ratios, or conditions usable to load balance incoming task execution requests across the internal compute capacity hosted by the first set of physical machines and the external compute capacity hosted by the second set of physical machines external to the cloud provider system, wherein the one or more threshold values, ratios, or conditions comprise a ratio of task execution requests to be maintained across the set of capacity providers such that the incoming task execution requests are split across the external compute capacity and the internal compute capacity in a way that maintains the ratio;
configure, using the one or more threshold values, ratios, or conditions, the cluster of compute capacity to load balance the incoming task execution requests across the internal compute capacity and the external compute capacity based at least in part on the one or more threshold values, ratios, or conditions;
subsequent to the configuration of the cluster, receive a first request to execute a first task in the cluster;
determine, based at least in part on the one or more threshold values, ratios, or conditions used to configure the cluster and one or more historical or current values, ratios, or conditions associated with the cluster, that the internal compute capacity provided by the first capacity provider of the set of capacity providers is to be used to execute the first task;
cause the first task to be executed using a first compute instance of the plurality of compute instances that is associated with the first capacity provider and hosted by the first set of physical machines of the cloud provider system;
receive a second request to execute a second task in the cluster;
determine, based at least in part on the one or more threshold values, ratios, or conditions used to configure the cluster and one or more updated historical or current values, ratios, or conditions associated with the cluster, that the external compute capacity provided by the second capacity provider of the set of capacity providers is to be used to execute the second task; and
cause the second task to be executed using a second compute instance hosted on or across the second set of physical machines external to the cloud provider system.
2 . The cloud provider system of claim 1 , wherein the first compute instance is associated with the cluster prior to the first request being received.
3 . The cloud provider system of claim 1 , wherein the first compute instance is acquired by the first capacity provider subsequent to the first request being received.
4 . The cloud provider system of claim 1 , wherein the container service is further configured to provide an application programming interface (API) usable to create a new rule or modify or delete an existing rule for load balancing incoming task execution requests across the first capacity provider and the second capacity provider.
5 . A computer-implemented method comprising:
providing a cluster of compute capacity and a set of capacity providers configured to provide compute capacity to the cluster for executing tasks in the cluster;
receiving, from a user computing device, one or more threshold values, ratios, or conditions usable to load balance incoming task execution requests across internal compute capacity hosted by a first set of physical machines of a cloud provider system and external compute capacity hosted by a second set of physical machines external to the cloud provider system, wherein the one or more threshold values, ratios, or conditions comprise a ratio of task execution requests to be maintained across the set of capacity providers such that the incoming task execution requests are split across the external compute capacity and the internal compute capacity in a manner that maintains the ratio;
configuring, using the one or more threshold values, ratios, or conditions, the cluster of compute capacity to load balance the incoming task execution requests across the internal compute capacity and the external compute capacity based at least in part on the one or more threshold values, ratios, or conditions;
subsequent to the configuration of the cluster, receiving, by a container service implemented by the cloud provider system, a first request to execute a first task in the cluster of compute capacity;
determining, by the container service and based at least in part on the one or more threshold values, ratios, or conditions used to configure the cluster and one or more historical or current values, ratios, or conditions associated with the cluster, that internal compute capacity hosted by a first set of physical machines of the cloud provider system implementing the container service is to be used to execute the first task;
executing the first task on a first compute instance hosted by the first set of physical machines of the cloud provider system;
receiving, by the container service, a second request to execute a second task in the cluster of compute capacity;
determining, by the container service and based at least in part on the one or more threshold values, ratios, or conditions used to configure the cluster and one or more updated historical or current values, ratios, or conditions associated with the cluster, that external compute capacity hosted by a second set of physical machines external to the cloud provider system is to be used to execute the second task; and
outputting an instruction to execute the second task on a second compute instance hosted by the second set of physical machines external to the cloud provider system.
6 . The computer-implemented method of claim 5 , wherein the first compute instance is associated with the cluster prior to the first request being received.
7 . The computer-implemented method of claim 5 , wherein the first compute instance is acquired by the first capacity provider subsequent to the first request being received.
8 . The computer-implemented method of claim 5 , further comprising providing an application programming interface (API) usable to create a new rule or modify or delete an rule for load balancing incoming task execution requests across the internal compute capacity and the external compute capacity.
9 . The computer-implemented method of claim 5 , further comprising determining that the first task is to be executed using internal compute capacity based at least in part on a usage of the internal compute capacity in the cluster having reached a threshold level.
10 . The computer-implemented method of claim 5 , wherein the first request includes a first indicator indicating that the first task is to be executed using internal compute capacity, and the second request includes a second indicator indicating that the second task is to be executed using external compute capacity.
11 . The computer-implemented method of claim 5 , wherein the first request includes an indicator indicating that the first task is to be executed using internal compute capacity that satisfies a first execution requirement, the method further comprising identifying the first compute instance in the cluster based at least in part on the first compute instance satisfying the first execution requirement.
12 . The computer-implemented method of claim 5 , wherein the first compute instance is one of a virtual machine instance, a bare-metal instance, a physical machine, a container, a node, or an offload card.
13 . A non-transitory computer-readable medium storing instructions that, when executed by a computing system within a cloud provider network, cause the computing system to perform operations comprising:
providing a cluster of compute capacity and a set of capacity providers configured to provide compute capacity to the cluster for executing tasks in the cluster;
receiving, from a user computing device, one or more threshold values, ratios, or conditions usable to load balance incoming task execution requests across internal compute capacity hosted by a first set of physical machines of a cloud provider system and external compute capacity hosted by a second set of physical machines external to the cloud provider system, wherein the one or more threshold values, ratios, or conditions comprise a ratio of task execution requests to be maintained across the set of capacity providers such that the incoming task execution requests are split across the external compute capacity and the internal compute capacity in a manner that maintains the ratio;
configuring, using the one or more threshold values, ratios, or conditions, the cluster of compute capacity to load balance the incoming task execution requests across the internal compute capacity and the external compute capacity based at least in part on the one or more threshold values, ratios, or conditions;
subsequent to the configuration of the cluster, receiving a first request to execute a first task in the cluster of compute capacity;
determining, based at least in part on the one or more threshold values, ratios, or conditions used to configure the cluster and one or more historical or current values, ratios, or conditions associated with the cluster, that internal compute capacity hosted by a first set of physical machines within the cloud provider network is to be used to execute the first task;
executing the first task on a first compute instance hosted by the first set of physical machines within the cloud provider network;
receiving a second request to execute a second task in the cluster of compute capacity;
determining, based at least in part on the one or more threshold values, ratios, or conditions used to configure the cluster and one or more updated historical or current values, ratios, or conditions associated with the cluster, that external compute capacity hosted by a second set of physical machines external to the cloud provider network is to be used to execute the second task; and
outputting an instruction to execute the second task on a second compute instance hosted by the second set of physical machines external to the cloud provider network.
14 . The non-transitory computer-readable medium of claim 13 , wherein the first compute instance is associated with the cluster prior to the first request being received.
15 . The non-transitory computer-readable medium of claim 13 , wherein the first compute instance is acquired by the first capacity provider subsequent to the first request being received.
16 . The non-transitory computer-readable medium of claim 13 , storing further instructions that, when executed by the computing system, cause the computing system to perform operations comprising providing an application programming interface (API) usable to create a new rule or modify or delete an existing rule for load balancing incoming task execution requests across the internal compute capacity and the external compute capacity according to a task execution rule associated with the cluster.
17 . The non-transitory computer-readable medium of claim 13 , storing further instructions that, when executed by the computing system, cause the computing system to perform operations comprising determining that the first task is to be executed using internal compute capacity based at least in part on a usage of the internal compute capacity in the cluster having reached a threshold level.
18 . The non-transitory computer-readable medium of claim 13 , wherein the first request includes a first indicator indicating that the first task is to be executed using internal compute capacity, and the second request includes a second indicator indicating that the second task is to be executed using external compute capacity.
19 . The non-transitory computer-readable medium of claim 13 , wherein the first request includes an indicator indicating that the first task is to be executed using internal compute capacity that satisfies a first execution requirement, the non-transitory computer-readable medium storing further instructions that, when executed by the computing system, cause the computing system to perform operations comprising identifying the first compute instance in the cluster based at least in part on the first compute instance satisfying the first execution requirement.
20 . The non-transitory computer-readable medium of claim 13 , wherein the first compute instance is one of a virtual machine instance, a bare-metal instance, a physical machine, a container, a node, or an offload card.
21 . The cloud provider system of claim 1 , wherein the external compute capacity belongs to a user associated with the user computing system.
22 . The computer-implemented method of claim 5 , wherein the external compute capacity belongs to a user associated with the user computing system.
23 . The non-transitory computer-readable medium of claim 13 , wherein the external compute capacity belongs to a user associated with the user computing system.
24 . The cloud provider system of claim 1 , wherein the one or more threshold values, ratios, or conditions further comprise a threshold resource cost such that a task execution request associated with a resource cost that exceeds the threshold resource cost is to be routed to the external compute capacity as opposed to the internal compute capacity.
25 . The cloud provider system of claim 1 , wherein the one or more threshold values, ratios, or conditions further comprise a threshold number of task execution requests to be reached before transitioning from routing task execution requests to the external compute capacity to routing them to the internal compute capacity.