Hierarchical networking for nested container clusters
Some embodiments of the invention provide a novel network architecture for deploying guest clusters (GCs) including workload machines for a tenant (or other entity) within an availability zone. The novel network architecture includes a virtual private cloud (VPC) deployed in the availability zone (AZ) that includes a centralized routing element that provides access to a gateway routing element of the AZ. In some embodiments, the centralized routing element provides a set of services for packets traversing a boundary of the VPC. The services, in some embodiments, include load balancing, firewall, quality of service (QoS) and may be stateful or stateless. Guest clusters are deployed within the VPC and use the centralized routing element of the VPC to access the gateway routing element of the AZ.
1. A method for deploying a plurality of guest clusters (GCs) for an entity in a datacenter comprising:
deploying a virtual private cloud (VPC) network for a first cluster of machines of the entity in the datacenter, the VPC network comprising a centralized routing element that provides access to a datacenter gateway routing element and provides a set of services for packets traversing a boundary of the first VPC; and
deploying, in the VPC network, a plurality of GCs and a GC network for each GC comprising a plurality of GC machines and a plurality of routing elements implementing a distributed routing element executing on a plurality of host computers along with GC machines, each GC network configured to use the VPC's centralized routing element to access the datacenter gateway routing element, wherein the GC comprises a set of service Pods for which a load balancer of the VPC provides a load balancing service,
wherein the set of service Pods connect to a network segment that is not directly reachable by the load balancer of the VPC, the load balancer of the VPC performs a first load balancing operation over a set of virtual machines (VMs) on which the Pods execute, and a VM in the set of VMs that receives a data message destined to a service Pod in the set of service Pods performs a second load balancing operation over the set of service Pods to select a service Pod in the set of service Pods and provide the data message to the selected service Pod.
2. The method of claim 1 , wherein each GC is a Kubernetes cluster.
3. The method of claim 1 , wherein the VPC is a Kubernetes cluster.
4. The method of claim 1 , wherein the VPC is a non-Kubernetes cluster comprising at least one of virtual machines and non-Kubernetes Pods.
5. The method of claim 1 , wherein a set of resources allocated to the VPC network are shared by the plurality of GC networks.
6. The method of claim 5 , wherein the shared resources comprise at least one of processing resources, storage resources, and network resources.
7. The method of claim 6 , wherein the shared resources comprise network resources and the network resources comprise a set of internet protocol (IP) addresses allocated to the VPC network.
8. The method of claim 1 , wherein the VPC network implements a distributed firewall comprising a set of firewall rules and each of the plurality of GC networks inherits the set of firewall rules for implementing the distributed firewall within the GC network.
9. The method of claim 1 , wherein the VPC network and a set of GC networks in the plurality of GC networks are deployed by a software defined datacenter manager that is aware of network addresses of each network component in the VPC network and the set of GC networks.
10. A system comprising:
a virtual private cloud (VPC) network for a first cluster of machines of an entity in a datacenter, the VPC network comprising a centralized routing element that provides access to a datacenter gateway routing element and provides a set of services for packets traversing a boundary of the first VPC; and
a plurality of guest clusters (GCs) and a GC network for each GC comprising a plurality of GC machines and a plurality of routing elements implementing a distributed routing element executing on a plurality of host computers along with GC machines, each GC network configured to use the VPC's centralized routing element to access the datacenter gateway routing element, wherein the GC comprises a set of service Pods for which a load balancer of the VPC provides a load balancing service,
wherein the set of service Pods connect to a network segment that is not directly reachable by the load balancer of the VPC, the load balancer of the VPC performs a first load balancing operation over a set of virtual machines (VMs) on which the Pods execute, and a VM in the set of VMs that receives a data message destined to a service Pod in the set of service Pods performs a second load balancing operation over the set of service Pods to select a service Pod in the set of service Pods and provide the data message to the selected service Pod.