Using CRDs to create externally routable addresses and route records for pods
Some embodiments provide an automated method for defining externally routable Pods within a Kubernetes cluster. In some embodiments, the Pod operates in a guest cluster has its own VPC (virtual private cloud) network in a datacenter with several other guest clusters that have their own VPC networks and their own set of managers. In some embodiments, a Pod within a GC can be made externally routable so that it can be directly addressable from an external client outside of the Pod's network by using two new Kubernetes CRDs (custom resource definitions), which are an IPPool CRD and a RouteSet CRD. Examples of such external clients include VMs or Pods in another GC or a supervisor cluster connected to the particular GC through a gateway, or from a machine outside of the network of all of the GCs or SC.
1 . A system comprising:
a first cluster comprising a plurality of worker nodes, each worker node being configured to host at least a network application, the first cluster being associated with a first virtual private cloud (VPC);
a supervisor cluster (SC) comprising a control plane, wherein the control plane is configured to:
detect a request to create a pool of addresses in response to a worker node addition in the first cluster;
allocate an externally routable subnet for the worker node;
define static routes between the pool of addresses and external networks through a gateway router associated with the SC; and
the gateway router configured to route traffic between the first cluster and an external network based on the static routes.
2 . The system of claim 1 , wherein the network application comprises a Pod, the first cluster comprises a Kubernetes cluster.
3 . The system of claim 1 , wherein the pool of addresses comprises an IP Pool.
4 . The system of claim 1 , wherein the control plane is further configured to manage middlebox service rules to enable operations on data packets exchanges between externally routable applications and external clients.
5 . The system of claim 1 , wherein the control plane is further configured to provide the externally routable subnet to a server.
6 . The system of claim 1 , wherein the control plane is further configured to provide a status update to a manager of the first cluster to update a routing record.
7 . The system of claim 1 , wherein the control plane is further configured to update an IP Pool customer resource instance using at least the externally routable subnet.
8 . The system of claim 1 , wherein the external network comprises a network of the SC.
9 . The system of claim 1 , wherein the first VPC comprises a set of managers to manage compute and network resources.
10 . The system of claim 1 , further comprising middlebox service rule to perform a middlebox service operation on packets sent between the first cluster and an external network based on the static routes.
11 . A method of deploying an externally routable network application in a first cluster associated with a first virtual private cloud (VPC), the method comprising:
detecting, at a control plane of a supervisor cluster (SC), a request to create a pool of addresses in response to the addition of a worker node within the first cluster;
allocating, by the control plane, an externally routable subnet for the worker node;
defining static routes between the pool of addresses and external networks via a gateway router associated with the SC; and
routing, by the gateway router, traffic between the first cluster and the external network based on the static routes.
12 . The method of claim 11 , wherein the network application comprises a Pod, and wherein deploying the externally routable network application further includes configuring the first cluster as a Kubernetes cluster.
13 . The method of claim 11 , wherein creating the pool of addresses includes creating an IP Pool to serve as the address pool for the network applications hosted by the worker nodes in the first cluster.
14 . The method of claim 11 , further comprising managing middlebox service rules at the control plane to enable operations on data packets exchanged between externally routable applications and external clients.
15 . The method of claim 11 , further comprising providing the allocated externally routable subnet to a server, wherein the server uses the externally routable subnet to configure the network application as accessible from the external network.
16 . The method of claim 11 , further comprising providing a status update to a manager of the first cluster, wherein the status update enables the manager to update a routing record for the network application hosted on the worker node.
17 . The method of claim 11 , further comprising updating an IP Pool custom resource instance in the first cluster with the externally routable subnet to enable external accessibility for the network application on the worker node.
18 . The method of claim 11 , wherein the external network includes a network associated with the supervisor cluster (SC) that allows communication between the first cluster and other clusters managed by the SC.
19 . The method of claim 11 , wherein managing resources in the first VPC includes managing compute and network resources by a set of managers associated with the first VPC.
20 . The method of claim 11 , further comprising enforcing a middlebox service rule at the gateway router, wherein the middlebox service rule performs a designated operation on packets exchanged between the first cluster and the external network based on the static routes.