Systems and methods for handling missing storage image layers while provisioning containers in computer clusters
The disclosed computer-implemented method for provisioning containers in computer clusters may include (1) identifying a request to provision a container on a host system within a cluster, (2) determining that a missing storage layer image is not locally stored at the host system and initiating a transfer of the missing storage layer image to the host system, (3) creating the container and executing an application within the container on the host system before the missing storage layer image has been transferred to the host system, (4) remotely resolving a read request by reading from an instance of the missing storage layer image that is stored at a separate host system within the cluster, and (5) resolving a subsequent read request by reading from an instance of the missing storage layer image that is locally stored at the host system. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for provisioning containers in computer clusters, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
identifying a request to provision a container on a host system within a cluster comprising a plurality of host systems, the container to comprise a plurality of storage layer images, each successive storage layer image within the plurality of storage layer images describing a set of differences applied to a storage view from within the container by the storage layer image;
determining, in response to the request, that a missing storage layer image within the plurality of storage layer images is not locally stored at the host system and initiating a transfer of the missing storage layer image to the host system;
creating the container on the host system and executing an application within the container on the host system before the missing storage layer image has been transferred to the host system;
receiving, from within the container, a read request that implicates the missing storage layer image and remotely resolving the read request by reading from an instance of the missing storage layer image that is stored at a separate host system within the cluster of host systems; and
resolving a subsequent read request that implicates the missing storage layer image after the missing storage layer image has been transferred to the host system by reading from an instance of the missing storage layer image that is locally stored at the host system.
2. The computer-implemented method of claim 1 , wherein the missing storage layer image is stored at each host system within a subset of the plurality of host systems.
3. The computer-implemented method of claim 1 , further comprising:
determining that a present storage layer image within the plurality of storage layer images is locally stored at the host system; and
resolving a read request that implicates the present storage layer image by reading from an instance of the present storage layer image locally stored at the host system.
4. The computer-implemented method of claim 1 , wherein the cluster comprises:
a compute tier that comprises the plurality of host systems and that provides primary storage for applications executing on the plurality of host systems; and
a data tier that comprises a plurality of data nodes and that provides secondary storage for data generated by the applications executing on the plurality of host systems.
5. The computer-implemented method of claim 4 , further comprising:
determining, in response to the request, that an additional missing storage layer image within the plurality of storage layer images is not available at any host system within the plurality of host systems; and
receiving, from within the container, an additional read request that implicates the additional missing storage layer image and remotely resolving the additional read request by reading from an instance of the additional missing storage layer image that is stored at a data node within the plurality of data nodes.
6. The computer-implemented method of claim 1 , further comprising:
determining, in response to the request, that an additional missing storage layer image within the plurality of storage layer images is not locally stored at the host system and initiating a transfer of the additional missing storage layer image to the host system; and
receiving, from within the container, an additional read request that implicates the additional missing storage layer image and remotely resolving the additional read request by reading from an instance of the additional missing storage layer image that is stored at a third host system within the cluster of host systems.
7. The computer-implemented method of claim 6 , further comprising selecting the third host system instead of the separate host system for resolving the additional read request to balance a load imposed on the separate host system and a load imposed on the third host system.
8. The computer-implemented method of claim 1 , further comprising receiving, at the host system, a manifest of storage layer images stored locally for each host system within the plurality of host systems.
9. The computer-implemented method of claim 1 , wherein the cluster implements storage reflection to store multiple instances of a volume across different host systems within the plurality of host systems when the volume is stored within the cluster.
10. The computer-implemented method of claim 9 , wherein the cluster determines how many instances of the volume to store within the cluster based at least in part on a type of the volume.
11. The computer-implemented method of claim 10 , wherein the cluster stores volumes comprising storage layer images at a greater rate than at least one additional type of volume that is also subject to storage reflection.
12. A system for provisioning containers in computer clusters, the system comprising:
an identification module, stored in memory, that identifies a request to provision a container on a host system within a cluster comprising a plurality of host systems, the container to comprise a plurality of storage layer images, each successive storage layer image within the plurality of storage layer images describing a set of differences applied to a storage view from within the container by the storage layer image;
a determination module, stored in memory, that determines, in response to the request, that a missing storage layer image within the plurality of storage layer images is not locally stored at the host system and initiates a transfer of the missing storage layer image to the host system;
a creation module, stored in memory, that creates the container on the host system and execute an application within the container on the host system before the missing storage layer image has been transferred to the host system;
a receiving module, stored in memory, that receives, from within the container, a read request that implicates the missing storage layer image and remotely resolves the read request by reading from an instance of the missing storage layer image that is stored at a separate host system within the cluster of host systems;
a resolving module, stored in memory, that resolves a subsequent read request that implicates the missing storage layer image after the missing storage layer image has been transferred to the host system by reading from an instance of the missing storage layer image that is locally stored at the host system; and
at least one physical processor configured to execute the identification module, the determination module, the creation module, the receiving module, and the resolving module.
13. The system of claim 12 , wherein the missing storage layer image is stored at each host system within a subset of the plurality of host systems.
14. The system of claim 12 , wherein:
the determination module further determines that a present storage layer image within the plurality of storage layer images is locally stored at the host system; and
the resolving module further resolves a read request that implicates the present storage layer image by reading from an instance of the present storage layer image locally stored at the host system.
15. The system of claim 12 , wherein the cluster comprises:
a compute tier that comprises the plurality of host systems and that provides primary storage for applications executing on the plurality of host systems; and
a data tier that comprises a plurality of data nodes and that provides secondary storage for data generated by the applications executing on the plurality of host systems.
16. The system of claim 15 , wherein:
the determination module further determines, in response to the request, that an additional missing storage layer image within the plurality of storage layer images is not available at any host system within the plurality of host systems; and
the receiving module further receives, from within the container, an additional read request that implicates the additional missing storage layer image and remotely resolves the additional read request by reading from an instance of the additional missing storage layer image that is stored at a data node within the plurality of data nodes.
17. The system of claim 12 , wherein:
the determination module further determines, in response to the request, that an additional missing storage layer image within the plurality of storage layer images is not locally stored at the host system and initiating a transfer of the additional missing storage layer image to the host system; and
the receiving module further receives, from within the container, an additional read request that implicates the additional missing storage layer image and remotely resolves the additional read request by reading from an instance of the additional missing storage layer image that is stored at a third host system within the cluster of host systems.
18. The system of claim 17 , wherein the receiving module selects the third host system instead of the separate host system for resolving the additional read request to balance a load imposed on the separate host system and a load imposed on the third host system.
19. The system of claim 12 , the determination module receives, at the host system, a manifest of storage layer images stored locally for each host system within the plurality of host systems.
20. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
identify a request to provision a container on a host system within a cluster comprising a plurality of host systems, the container to comprise a plurality of storage layer images, each successive storage layer image within the plurality of storage layer images describing a set of differences applied to a storage view from within the container by the storage layer image;
determine, in response to the request, that a missing storage layer image within the plurality of storage layer images is not locally stored at the host system and initiate a transfer of the missing storage layer image to the host system;
create the container on the host system and executing an application within the container on the host system before the missing storage layer image has been transferred to the host system;
receive, from within the container, a read request that implicates the missing storage layer image and remotely resolve the read request by reading from an instance of the missing storage layer image that is stored at a separate host system within the cluster of host systems; and
resolve a subsequent read request that implicates the missing storage layer image after the missing storage layer image has been transferred to the host system by reading from an instance of the missing storage layer image that is locally stored at the host system.