Layer-aware data movement control for containers
An apparatus in one embodiment comprises a plurality of container host devices of at least one processing platform. The container host devices implement a plurality of containers for executing applications on behalf of one or more tenants of cloud infrastructure. The containers have associated layer structures each characterizing container images of respective different ones of the containers. Movement of container data between different storage devices by at least one of the container host devices for at least one of the containers is controlled based at least in part on one or more characteristics of a corresponding one of the layer structures. For example, controlling movement of container data between the different storage devices may comprise assigning at least one of different prefetching priority weights and different cache-swapping priority weights to different layers of the given layer structure.
1. An apparatus comprising:
a plurality of container host devices;
the container host devices implementing a plurality of containers for executing applications on behalf of one or more tenants of cloud infrastructure;
the containers having associated layer structures each characterizing container images of respective different ones of the containers;
wherein movement of container data between different storage devices by at least one of the container host devices for at least one of the containers is controlled based at least in part on one or more characteristics of a corresponding one of the layer structures; and
wherein controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of the given layer structure comprises establishing a prefetching priority in which the container host device prefetches container data from an external storage system into persistent storage of the container host device starting from a read-write layer at a highest priority for prefetching from the external storage system and moving upward through the given layer structure with decreasing priority for successive ones of a plurality of read-only layers;
the container host devices being implemented on at least one processing platform comprising a processor coupled to a memory.
2. The apparatus of claim 1 wherein at least a subset of the container host devices comprise respective virtual machines controlled by a hypervisor of the processing platform.
3. The apparatus of claim 1 wherein the container host devices are configured as a plurality of nodes each sharing a common storage system external to the container host devices.
4. The apparatus of claim 1 wherein the different storage devices comprise at least portions of respective persistent storage of a given one of the container host devices and an external storage system shared by the container host devices.
5. The apparatus of claim 1 wherein at least one of the container host devices is configured to control movement of container data between different storage devices by swapping at least a portion of the container data stored in a cache of the container host device with one or more other portions of container data stored in another storage device based at least in part on one or more characteristics of a corresponding one of the layer structures.
6. The apparatus of claim 1 wherein a given one of the layer structures comprises:
one or more read-only layers each associated with one or more of the container images; and
one or more read-write layers each associated with at least one running instance of at least one of the containers.
7. The apparatus of claim 1 further comprising an acceleration module implemented in the container host device and configured to generate a plurality of prefetch worker threads for prefetching respective portions of the container data in accordance with the established prefetch priority.
8. The apparatus of claim 1 wherein controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of the given layer structure comprises establishing a cache-swapping priority in which the container host device controls movement of container data between a cache of the container host device and another storage device starting from a read-write layer at a highest priority for remaining in the cache and moving upward through the given layer structure with decreasing priority for successive ones of a plurality of read-only layers.
9. The apparatus of claim 8 further comprising an acceleration module implemented in the container host device and configured to generate a plurality of cache paging threads for cache-swapping respective portions of the container data in accordance with the established cache-swapping priority.
10. The apparatus of claim 8 wherein within a particular one of the read-only and read-write layers of the given layer structure, the container host device utilizes one of a least recently used policy and a least frequently used policy applied solely within that particular layer to control movement of container data between the cache of the container host device and the other storage device.
11. The apparatus of claim 1 wherein controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of the given layer structure comprises at least one of:
assigning different prefetching priority weights to different layers of the given layer structure; and
assigning different cache-swapping priority weights to different layers of the given layer structure.
12. The apparatus of claim 1 wherein controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of the given layer structure comprises maintaining separate least recently used or least frequency used lists for each of two or more distinct layers of the given layer structure.
13. A method comprising:
configuring a plurality of container host devices in cloud infrastructure;
utilizing the container host devices to implement a plurality of containers for executing applications on behalf of one or more tenants of the cloud infrastructure;
obtaining layer structures each characterizing container images of respective different ones of the containers; and
controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of a corresponding one of the layer structures;
wherein controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of the given layer structure comprises establishing a prefetching priority in which the container host device prefetches container data from an external storage system into persistent storage of the container host device starting from a read-write layer at a highest priority for prefetching from the external storage system and moving upward through the given layer structure with decreasing priority for successive ones of a plurality of read-only layers;
the method being performed by at least one processing platform comprising a processor coupled to a memory.
14. The method of claim 13 wherein controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of the given layer structure comprises establishing a cache-swapping priority in which the container host device controls movement of container data between a cache of the container host device and another storage device starting from a read-write layer at a highest priority for remaining in the cache and moving upward through the given layer structure with decreasing priority for successive ones of a plurality of read-only layers.
15. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing platform comprising a processor coupled to a memory causes the processing platform:
to configure a plurality of container host devices in cloud infrastructure;
to utilize the container host devices to implement a plurality of containers for executing applications on behalf of one or more tenants of the cloud infrastructure;
to obtain layer structures each characterizing container images of respective different ones of the containers; and
to control movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of a corresponding one of the layer structures;
wherein controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of the given layer structure comprises establishing a prefetching priority in which the container host device prefetches container data from an external storage system into persistent storage of the container host device starting from a read-write layer at a highest priority for prefetching from the external storage system and moving upward through the given layer structure with decreasing priority for successive ones of a plurality of read-only layers.
16. The computer program product of claim 15 wherein controlling movement of container data between different storage devices by at least one of the container host devices for at least one of the containers based at least in part on one or more characteristics of the given layer structure comprises establishing a cache-swapping priority in which the container host device controls movement of container data between a cache of the container host device and another storage device starting from a read-write layer at a highest priority for remaining in the cache and moving upward through the given layer structure with decreasing priority for successive ones of a plurality of read-only layers.
17. The computer program product of claim 15 wherein at least one of the container host devices is configured to control movement of container data between different storage devices by swapping at least a portion of the container data stored in a cache of the container host device with one or more other portions of container data stored in another storage device based at least in part on one or more characteristics of a corresponding one of the layer structures.
18. The computer program product of claim 15 wherein at least a subset of the container host devices comprise respective virtual machines controlled by a hypervisor of the processing platform.
19. The computer program product of claim 15 wherein the container host devices are configured as a plurality of nodes each sharing a common storage system external to the container host devices.
20. The computer program product of claim 15 wherein the different storage devices comprise at least portions of respective persistent storage of a given one of the container host devices and an external storage system shared by the container host devices.