Ephemeral storage management for container-based virtual machines
A virtualized computing system includes: a host cluster including hosts executing a virtualization layer on hardware platforms thereof, the virtualization layer configured to support execution of virtual machines (VMs), the VMs including a pod VM, the pod VM including a container engine configured to support execution of containers in the pod VM, the pod VM including a first virtual disk attached thereto; and an orchestration control plane integrated with the virtualization layer, the orchestration control plane including a master server in communication with a pod VM controller, the pod VM controller configured to execute in the virtualization layer external to the VMs and cooperate with a pod VM agent in the pod VM, the pod VM agent generating root directories for the containers in the pod VM, each of the root directories comprising a union a read/write ephemeral layer stored on the first virtual disk and a read-only layer.
1. A virtualized computing system, comprising:
a host cluster including hosts executing hypervisors directly on hardware platforms thereof, the hypervisors configured to support execution of virtual machines (VMs), the VMs including a pod VM, the pod VM including a container engine configured to support execution of containers in the pod VM, the pod VM including a first virtual disk attached thereto; and
an orchestration control plane integrated with the hypervisors, the orchestration control plane including a master server in communication with a pod VM controller, the pod VM controller configured to execute in a first hypervisor of the hypervisors external to the VMs and configured to cooperate with a pod VM agent in the pod VM, the pod VM agent configured to generate root directories for the containers in the pod VM, each of the root directories comprising a union of a read/write ephemeral layer and a read-only layer, the read/write ephemeral layer of each of the root directories stored on the first virtual disk.
2. The virtualized computing system of claim 1 , wherein the first virtual disk has a filesystem with directories, each of the directories comprising the read/write ephemeral layer of a respective one of the root directories.
3. The virtualized computing system of claim 2 , wherein the filesystem of the first virtual disk specifies a storage quota for a first directory of the directories.
4. The virtualized computing system of claim 3 , wherein the pod VM agent is configured to set the storage quota for the first directory in response to at least one of a reservation and a limit, received from the master server, for a first container of the containers, the first directory comprising the read/write ephemeral layer of a first root directory of the root directories for the first container.
5. The virtualized computing system of claim 1 , further comprising:
second virtual disks storing container images for the containers, where the read-only layer of each of the root directories includes a directory tree formed from one or more image layers stored on a respective one of the second virtual disks.
6. The virtualized computing system of claim 1 , wherein the master server is configured to provision the first virtual disk and size the first virtual disk in response to at least one of a reservation of ephemeral storage, a limit of ephemeral storage, or a default amount of ephemeral storage for each of the containers.
7. The virtualized computing system of claim 6 , wherein the master server is configured to size the first virtual disk further in response to a size of an ephemeral volume shared by the containers.
8. A method of managing ephemeral storage for containers executing in a virtualized computing system, the virtualized computing system including an orchestration control plane and a host cluster, the host cluster having hosts executing hypervisors on hardware platforms thereof, the hypervisors configured to support execution of virtual machines (VMs), the orchestration control plane including a master server in communication with a pod VM controller, the pod VM controller configured to execute in a first hypervisor of the hypervisors external to the VMs, the method comprising:
identifying, by the pod VM controller, deployment of a pod VM of the VMs, the pod VM including a container engine configured to support execution of the containers in the pod VM, the pod VM including a first virtual disk attached thereto, the pod VM including a pod VM agent configured to cooperate with the pod VM controller; and
generating, by the pod VM agent, root directories for the containers in the pod VM, each of the root directories comprising a union of a read/write ephemeral layer and a read-only layer, the read/write ephemeral layer for each of the root directories stored on the first virtual disk.
9. The method of claim 8 , wherein the first virtual disk has a filesystem with directories, each of the directories comprising the read/write ephemeral layer of a respective one of the root directories.
10. The method of claim 9 , wherein the filesystem of the first virtual disk specifies a storage quota for a first directory of the directories.
11. The method of claim 10 , further comprising:
setting, by the pod VM agent, the storage quota for the first directory in response to at least one of a reservation and a limit, received from the master server, for a first container of the containers, the first directory comprising the read/write ephemeral layer of a first root directory of the root directories for the first container.
12. The method of claim 8 , further comprising:
attaching, by the pod VM controller to the pod VM, second virtual disks that store container images for the containers, where the read-only layer of each of the root directories includes a directory tree formed from one or more image layers stored on a respective one of the second virtual disks.
13. The method of claim 8 , further comprising:
detecting, by the pod VM controller, termination of the pod VM; and
mounting a filesystem of the first virtual disk for access by a system VM executing in the host cluster.
14. The method of claim 8 , further comprising:
detecting, by the pod VM controller, termination of the pod VM; and
deleting the first virtual disk.
15. A non-transitory computer readable medium comprising instructions to be executed in a computing device to cause the computing device to carry out a method of managing ephemeral storage for containers executing in a virtualized computing system, the virtualized computing system including an orchestration control plane and a host cluster, the host cluster having hosts executing hypervisors on hardware platforms thereof, the hypervisors configured to support execution of virtual machines (VMs), the orchestration control plane including a master server in communication with a pod VM controller, the pod VM controller configured to execute in a first hypervisor of the hypervisors external to the VMs, the method comprising:
identifying, by the pod VM controller, deployment of a pod VM of the VMs, the pod VM including a container engine configured to support execution of the containers in the pod VM, the pod VM including a first virtual disk attached thereto, the pod VM including a pod VM agent configured to cooperate with the pod VM controller; and
generating, by the pod VM agent, root directories for the containers in the pod VM, each of the root directories comprising a union of a read/write ephemeral layer and a read-only layer, the read/write ephemeral layer for each of the root directories stored on the first virtual disk.
16. The non-transitory computer readable medium of claim 15 , wherein the first virtual disk has a filesystem with directories, each of the directories comprising the read/write ephemeral layer of a respective one of the root directories.
17. The non-transitory computer readable medium of claim 16 , wherein the filesystem of the first virtual disk specifies a storage quota for a first directory of the directories.
18. The non-transitory computer readable medium of claim 17 , further comprising:
setting, by the pod VM agent, the storage quota for the first directory in response to at least one of a reservation and a limit, received from the master server, for a first container of the containers, the first directory comprising the read/write ephemeral layer of a first root directory of the root directories for the first container.
19. The non-transitory computer readable medium of claim 15 , further comprising:
attaching, by the pod VM controller to the pod VM, second virtual disks that store container images for the containers, where the read-only layer of each of the root directories includes a directory tree formed from one or more image layers stored on a respective one of the second virtual disks.
20. The non-transitory computer readable medium of claim 15 , further comprising:
detecting, by the pod VM controller, termination of the pod VM; and
mounting a filesystem of the first virtual disk for access by a system VM executing in the host cluster.