Hybrid approach to performing a lazy pull of container images
Embodiments of the present disclosure provide a hybrid approach to performing a lazy pull of a container image. A file system in user space (FUSE) is utilized to lazy pull the container image, and manage file requests from the container while the container image is being fetched locally. During the retrieving, the FUSE may receive from the container, one or more file requests, and may temporarily block each of the one or more file requests until it can process them. Once the container image is fully fetched locally, the overlay structure of the container image expected by a file system in the kernel (e.g., Overlay FS) is created and control is passed to the file system in the kernel. The FUSE may then unmount itself, to expose the container to the underlying mount point.
1. A method comprising:
retrieving, by a processing device and a first file system in user space (FUSE), a plurality of files corresponding to an image file to build, in a mount point implemented in an operating system kernel, one or more base layers for running a container;
during the retrieving and while the one or more base layers are being built, receiving, by the processing device, from the container, one or more file requests, wherein the FUSE acts as a proxy between the container and the image file;
processing, by the processing device, using the FUSE, each of the one or more file requests by:
determining, by the processing device, whether the requested file is one of the plurality of files;
in response to determining that the requested file is one of the plurality of files, retrieving, by the processing device, based on whether the requested file has been retrieved by the FUSE, the requested file from the mount point or retrieving, by the processing device, the requested file from a registry server and writing, by the processing device, the requested file to the mount point; and
providing, by the processing device, the requested file to the container; and
in response to determining that all of the plurality of files have been retrieved or requested:
processing, by the processing device, any outstanding file requests; and
instructing, by the processing device, the mount point to run the container using the one or more base layers built in the mount point.
2. The method of claim 1 , wherein the mount point is initially generated with a set of empty layers, and wherein the FUSE is mounted at a FUSE mount point that is overlaid on top of the mount point as part of a second file system implemented in the operating system kernel.
3. The method of claim 2 , wherein the mount point and the FUSE mount point are combined into a single mount point by the second file system implemented in the operating system kernel.
4. The method of claim 2 , further comprising:
unmounting, by the processing device, the FUSE to expose the container to the mount point comprising the one or more base layers.
5. The method of claim 1 , wherein:
in response to determining that the requested file has been retrieved by the FUSE, retrieving, by the processing device, the requested file from the mount point; and
in response to determining that the requested file has not been retrieved by the FUSE, retrieving, by the processing device, the requested file from the registry server and writing the requested file to the mount point.
6. The method of claim 5 , wherein processing a file request further comprises:
in response to determining that the requested file is not one of the plurality of files, creating, by the processing device, an empty file in a write layer of the mount point.
7. The method of claim 1 , wherein the FUSE retrieves the plurality of files as a background process.
8. A system comprising:
a memory; and
a processing device, operatively coupled to the memory, the processing device to:
retrieve, by a first file system in user space (FUSE), a plurality of files corresponding to an image file to build, in a mount point implemented in an operating system kernel, one or more base layers for running a container;
during the retrieving and while the one or more base layers are being built, receive from the container, one or more file requests, wherein the FUSE acts as a proxy between the container and the image file;
process, using the FUSE, each of the one or more file requests by:
determining whether the requested file is one of the plurality of files;
in response to determining that the requested file is one of the plurality of files, retrieving, based on whether the requested file has been retrieved by the FUSE, the requested file from the mount point or retrieving the requested file from a registry server and writing the requested file to the mount point; and
providing the requested file to the container; and
in response to determining that all of the plurality of files have been retrieved or requested:
process any outstanding file requests; and
instruct the mount point to run the container using the one or more base layers built in the mount point.
9. The system of claim 8 , wherein the mount point is initially generated with a set of empty layers, and wherein the FUSE is mounted at a FUSE mount point that is overlaid on top of the mount point as part of a second file system implemented in the operating system kernel.
10. The system of claim 9 , wherein the mount point and the FUSE mount point are combined into a single mount point by the second file system implemented in the operating system kernel.
11. The system of claim 9 , wherein the processing device is further to:
unmount the FUSE to expose the container to the mount point comprising the one or more base layers.
12. The system of claim 8 , wherein:
in response to determining that the requested file has been retrieved by the FUSE, the processing device retrieves the requested file from the mount point; and
in response to determining that the requested file has not been retrieved by the FUSE, the processing device retrieves the requested file from the registry server and writes the requested file to the mount point.
13. The system of claim 12 , wherein to process a file request, the processing device is further to:
in response to determining that the requested file is not one of the plurality of files, create an empty file in a write layer of the mount point.
14. The system of claim 8 , wherein the FUSE retrieves the plurality of files as a background process.
15. A non-transitory machine-readable storage medium having instructions stored thereon which, when executed by a processing device, cause the processing device to:
retrieve, by a first file system in user space (FUSE), a plurality of files corresponding to an image file to build, in a mount point implemented in an operating system kernel, one or more base layers for running a container;
during the retrieving and while the one or more base layers are being built, receive from the container, one or more file requests, wherein the FUSE acts as a proxy between the container and the image file;
process, using the FUSE, each of the one or more file requests by:
determining whether the requested file is one of the plurality of files;
in response to determining that the requested file is one of the plurality of files, retrieving, based on whether the requested file has been retrieved by the FUSE, the requested file from the mount point or retrieving the requested file from a registry server and writing the requested file to the mount point; and
providing the requested file to the container; and
in response to determining that all of the plurality of files have been retrieved or requested:
process any outstanding file requests; and
instruct the mount point to run the container using the one or more base layers built in the mount point.
16. The non-transitory machine-readable storage medium of claim 15 , wherein the mount point is initially generated with a set of empty layers, and wherein the FUSE is mounted at a FUSE mount point that is overlaid on top of the mount point as part of a second file system implemented in the operating system kernel.
17. The non-transitory machine-readable storage medium of claim 16 , wherein the mount point and the FUSE mount point are combined into a single mount point by the second file system implemented in the operating system kernel.
18. The non-transitory machine-readable storage medium of claim 16 , wherein the processing device is further to:
unmount the FUSE to expose the container to the mount point comprising the one or more base layers.
19. The non-transitory machine-readable storage medium of claim 15 , wherein:
in response to determining that the requested file has been retrieved by the FUSE, the processing device retrieves the requested file from the mount point; and
in response to determining that the requested file has not been retrieved by the FUSE, the processing device retrieves the requested file from the registry server and writes the requested file to the mount point.
20. The non-transitory machine-readable storage medium of claim 19 , wherein to process a file request, the processing device is further to:
in response to determining that the requested file is not one of the plurality of files, create an empty file in a write layer of the mount point.