Intelligent layer control of redundant content in container images
Intelligent layer control for building image files into new container images while avoiding redundant files stored between read-write and read-only image layers. Pre-commit processing module (PPM) filters updated files. Hash values of updated files are compared with hash values of original files. Where hash values are the same, PPM generates an OMD file that updates the inode metadata of original files with metadata of the updated file and removes the updated file. Hash values that are not the same, with differ values less than a threshold value, PPM generates an OMD file with updates to the inode metadata, content change flag and/or delta content info reflecting the updated file and removes updated file from container file system. OMD files are passed to storage driver whereby metadata and/or changes to content are overridden and merged with original file content generating a single unified user view of the container file system.
1. A computer-implemented method for building or modifying an image file into a new image for deployment as a container, the computer-implemented method comprising:
inspecting, by a daemon, files or directories of a container file system of the image file for changes to the files or the directories;
in response to inspection of the container file system containing changes to the files or the directories, filtering, by the daemon, an updated file copied-up from the image file into a read-write layer;
comparing, by the daemon, a hash value of an original file of the container file system from a lower, read-only layer of the image file with a hash value of the updated file;
in response to comparing the hash value of the original file with the hash value of the updated file, the hash value of the original file and the hash value of the updated file are the same or a difference between the hash value of the original file and the hash value of the updated file exceeds a default differ value, generating, by the daemon, an override and merge data (OMD) file comprising an override block and a merge block;
updating, by the daemon, metadata of the override block from inode metadata of the original file stored in the lower layer of the image file to metadata of the updated file; and
removing, by the daemon, the updated file from the container file system.
2. The computer-implemented method of claim 1 , wherein the difference between the hash value of the original file and the hash value of the updated file exceeds a default differ value, the computer-implemented method further comprises:
updating, by the daemon, the merge block of the OMD file with delta content information indicating content has changed between the original file and the updated file, and a pointer indicating a storage area containing delta content.
3. The computer-implemented method of claim 1 , further comprising:
scanning, by the storage driver, the read-write layer for each file present within the read-write layer;
upon detecting the OMD file within the read-write layer, parsing, by the storage driver, the OMD file for a content change flag within the merge block; and
upon parsing the OMD file and identifying the content change flag within the OMD file has a value of true, merging, by the storage driver, content of the original file with delta content of the OMD file.
4. The computer-implemented method of claim 3 , further comprising:
upon parsing the OMD file and identifying the content change flag within the OMD file has a value of false, overriding, by the storage driver, the inode metadata of the original file with metadata of the override block of the OMD file.
5. The computer-implemented method of claim 3 , further comprising:
overriding, by the storage driver, the inode metadata of the original file with metadata of the override block of the OMD file.
6. The computer-implemented method of claim 3 , further comprising:
running, by the daemon, a commit operation creating the new image comprising a new read-only layer containing the files of read-write layer placed on top of all previous read-only layers from the image file.
7. The computer-implemented method of claim 1 , wherein generating the OMD file comprising the override block and the merge block occurs when no content change between the original file and the updated file occurs, only a change to one or more properties of the inode metadata of the original file stored in the lower layer, or there is a change to the between the original file and the updated file and a change in file size below a threshold level.
8. A computing program product for building or modifying image files into a new image for deployment as a container comprising:
one or more computer readable storage media having computer-readable program instructions stored on the one or more computer readable storage media, said program instructions executes a computer-implemented method comprising:
inspecting, by a daemon, files or directories of a container file system of the image file for changes to the files or the directories;
in response to inspection of the container file system containing changes to the files or the directories, filtering, by the daemon, an updated file copied-up from the image file into a read-write layer;
comparing, by the daemon, a hash value of an original file of the container file system from a lower, read-only layer of the image file with a hash value of the updated file;
in response to comparing the hash value of the original file with the hash value of the updated file, the hash value of the original file and the hash value of the updated file are the same or a difference between the hash value of the original file and the hash value of the updated file exceeds a default differ value, generating, by the daemon, an override and merge data (OMD) file comprising an override block and a merge block;
updating, by the daemon, metadata of the override block from inode metadata of the original file stored in the lower layer of the image file to metadata of the updated file; and
removing, by the daemon, the updated file from the container file system.
9. The computing program product of claim 8 , wherein the difference between the hash value of the original file and the hash value of the updated file exceeds a default differ value, the computer-implemented method further comprises:
updating, by the daemon, the merge block of the OMD file with delta content information indicating content has changed between the original file and the updated file, and a pointer indicating a storage area containing delta content.
10. The computing program product of claim 8 , further comprising:
placing, by a storage driver of the daemon, the OMD file in the read-write layer;
scanning, by the storage driver, the read-write layer for each file present within the read-write layer;
upon detecting the OMD file within the read-write layer, parsing, by the storage driver, the OMD file for a content change flag within the merge block; and
upon parsing the OMD file and identifying the content change flag within the OMD file has a value of true, merging, by the storage driver, content of the original file with delta content of the OMD file and place the content and delta content into a final file within a user viewable layer of the image file.
11. The computing program product of claim 10 , further comprising:
upon parsing the OMD file and identifying the content change flag within the OMD file has a value of false, overriding, by the storage driver, the inode metadata of the original file with metadata of the override block of the OMD file.
12. The computing program product of claim 10 , further comprising:
overriding, by the storage driver, the inode metadata of the original file with metadata of the override block of the OMD file.
13. The computing program product of claim 10 , further comprising:
running, by the daemon, a commit operation creating the new image comprising a new read-only layer containing the files of read-write layer placed on top of all previous read-only layers from the image file.
14. The computing program product of claim 8 , wherein generating the OMD file comprising the override block and the merge block occurs when no content change between the original file and the updated file occurs, only a change to one or more properties of the inode metadata of the original file stored in the lower layer, or there is a change to the content between the original file and the updated file and a change in file size below a threshold level.
15. A computer system for building or modifying image files into a new image for deployment as a container comprising:
a processor; and
a computer-readable storage media coupled to the processor, wherein the computer-readable storage media contains program instructions executing, via the processor, a computer-implemented method comprising:
inspecting, by a daemon, files or directories of a container file system of the image file for changes to the files or the directories;
in response to inspection of the container file system containing changes to the files or the directories, filtering, by the daemon, an updated file copied-up from the image file into a read-write layer;
comparing, by the daemon, a hash value of an original file of the container file system from a lower, read-only layer of the image file with a hash value of the updated file;
in response to comparing the hash value of the original file with the hash value of the updated file, the hash value of the original file and the hash value of the updated file are the same or a difference between the hash value of the original file and the hash value of the updated file exceeds a default differ value, generating, by the daemon, an override and merge data (OMD) file comprising an override block and a merge block;
updating, by the daemon, metadata of the override block from inode metadata of the original file stored in the lower layer of the image file to metadata of the updated file; and
removing, by the daemon, the updated file from the container file system.
16. The computer system of claim 15 , wherein the difference between the hash value of the original file and the hash value of the updated file exceeds a default differ value, the computer-implemented method further comprises:
updating, by the daemon, the merge block of the OMD file with delta content information indicating content has changed between the original file and the updated file, and a pointer indicating a storage area containing delta content.
17. The computer system of claim 15 , further comprising:
placing, by a storage driver of the daemon, the OMD file in the read-write layer;
scanning, by the storage driver, the read-write layer for each file present within the read-write layer;
upon detecting the OMD file within the read-write layer, parsing, by the storage driver, the OMD file for a content change flag within the merge block; and
upon parsing the OMD file and identifying the content change flag within the OMD file has a value of true, merging, by the storage driver, content of the original file with delta content of the OMD file.
18. The computer system of claim 17 , further comprising:
upon parsing the OMD file and identifying the content change flag within the OMD file has a value of false, overriding, by the storage driver, the inode metadata of the original file with metadata of the override block of the OMD file.
19. The computer system of claim 17 , further comprising:
overriding, by the storage driver, the inode metadata of the original file with metadata of the override block of the OMD file.
20. The computer system of claim 15 , wherein generating the OMD file comprising the override block and the merge block occurs when no content change between the original file and the updated file occurs, only a change to one or more properties of the inode metadata of the original file stored in the lower layer, or there is a change to the between the original file and the updated file and a change in file size below a threshold level.