Method and system for semantic metadata compression in a two-tier storage system using copy-on-write
Example embodiments of the present invention relate to a method, a system, and a computer program product for managing metadata. The method includes determining a common characteristic of metadata entries for a plurality of files in a directory of a file system and computing a representative metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries. The plurality of metadata entries in the file system then may be represented according to the computed representative metadata entry.
1. A computer-implemented method for compressing metadata entries for a plurality of files in a directory for a file system comprising:
determining a common characteristic of metadata entries for a plurality of files in a directory of a file system in a distributed storage system having a distributed metadata store, wherein the distributed storage system has a plurality of tiers having a plurality of dynamically-loadable namespaces, wherein the plurality of files in the directory are represented as a respective plurality of objects, and wherein the metadata entries for the plurality of files in the directory of the file system are file names for the plurality of files in the directory of the file system;
computing a representative metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries;
representing the plurality of metadata entries in the file system according to the computed representative metadata entry;
replacing values in the metadata entries for the plurality of files in the directory of the file system with a same approximated value;
compressing the metadata entries for the plurality of files in the directory of the file system;
receiving a command to create a modified file name for a modified file in the directory of the file system; and
performing a copy-on-write for the modified file name entry for the modified file in the directory of the file system.
2. The method of claim 1 wherein computing a representative metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries comprises computing a single metadata entry as a regular expression.
3. The method of claim 2 wherein the metadata entries for the plurality of files in the directory have a sequentially-ordered set of i-node numbers.
4. The method of claim 1 wherein replacing values in the metadata entries further comprises replacing timestamp values in the metadata entries for the plurality of files in the directory of the file system with a same approximated timestamp value.
5. The method of claim 1 wherein the plurality of files in the directory of the file system are stored in a fast tier of storage of a storage system, the method further comprising:
storing the plurality of objects representing the plurality of files in the directory of the file system in a capacity tier of object storage of the storage system.
6. The method of claim 5 further comprising:
allocating a sequential range of i-nodes in the file system in the fast tier of storage of the storage system; and
restoring the plurality of files represented as the plurality of objects in the capacity tier of object storage of the storage system to the file system in the fast tier of storage of the storage system according to the common characteristics of the metadata entries for the plurality of files.
7. The method of claim 6 wherein restoring the plurality of files represented as the plurality of objects in the capacity tier of object storage of the storage system to the file system in the fast tier of storage of the storage system according to the common characteristics of the metadata entries for the plurality of files comprises:
computing the file names for the plurality of files according to the representative metadata entry and the common characteristic of the metadata entries; and
assigning to each i-node in the sequential range of i-nodes an object from the capacity tier of object storage in the storage system according to its respective computed file name.
8. A distributed storage system for compressing metadata entries for a plurality of files in a directory for a file system comprising:
a processor; and
memory storing computer program code that when executed on the processor causes the system to manage metadata by performing the operation of:
determining a common characteristic of metadata entries for a plurality of files in a directory of a file system in the distributed storage system having a distributed metadata store, wherein the distributed storage system has a plurality of tiers having a plurality of dynamically-loadable namespaces, wherein the plurality of files in the directory are represented as a respective plurality of objects, and wherein the metadata entries for the plurality of files in the directory of the file system are file names for the plurality of files in the directory of the file system;
computing a representative metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries;
representing the plurality of metadata entries in the file system according to the computed representative metadata entry;
replacing values in the metadata entries for the plurality of files in the directory of the file system with a same approximated value;
compressing the metadata entries for the plurality of files in the directory of the file system;
receiving a command to create a modified file name for a modified file in the directory of the file system; and
performing a copy-on-write for the modified file name entry for the modified file in the directory of the file system.
9. The system of claim 8 wherein computing a metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries comprises computing a single metadata entry as a regular expression.
10. The system of claim 9 wherein the metadata entries for the plurality of files in the directory have a sequentially-ordered set of i-node numbers.
11. The system of claim 8 wherein replacing values in the metadata entries further comprises replacing timestamp values in the metadata entries for the plurality of files in the directory of the file system with a same approximated timestamp value.
12. The system of claim 8 , wherein the plurality of files in the directory of the file system are stored in a fast tier of storage of a storage system, the method further comprising: storing the plurality of objects representing the plurality of files in the directory of the file system in a capacity tier of object storage of the storage system.
13. The system of claim 12 further comprising:
allocating a sequential range of i-nodes in the file system in the fast tier of storage of the storage system; and
restoring the plurality of files represented as the plurality of objects in the capacity tier of object storage of the storage system to the file system in the fast tier of storage of the storage system according to the common characteristics of the metadata entries for the plurality of files.
14. The system of claim 13 wherein restoring the plurality of files represented as the plurality of objects in the capacity tier of object storage of the storage system to the file system in the fast tier of storage of the storage system according to the common characteristics of the metadata entries for the plurality of files comprises:
computing the file names for the plurality of files according to the representative metadata entry and the common characteristic of the metadata entries; and
assigning to each i-node in the sequential range of i-nodes an object from the capacity tier of object storage in the storage system according to its respective computed file name.
15. A computer program product including a non-transitory computer readable storing medium having computer program code encoded thereon that when executed on a processor of a computer causes the computer to compress metadata entries for a plurality of files in a directory for a file system, the computer program code comprising:
computer program code for determining a common characteristic of metadata entries for a plurality of files in a directory of a file system in a distributed storage system having a distributed metadata store, wherein the distributed storage system has a plurality of tiers having a plurality of dynamically-loadable namespaces, wherein the plurality of files in the directory are represented as a respective plurality of objects, and wherein the metadata entries for the plurality of files in the directory of the file system are file names for the plurality of files in the directory of the file system;
computer program code for computing a representative metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries;
computer program code for representing the plurality of metadata entries in the file system according to the computed representative metadata entry;
computer program code for replacing values in the metadata entries for the plurality of files in the directory of the file system with a same approximated value;
computer program code for compressing the metadata entries for the plurality of files in the directory of the file system;
computer program code for receiving a command to create a modified file name for a modified file in the directory of the file system; and
computer program code for performing a copy-on-write for the modified file name entry for the modified file in the directory of the file system.
16. The computer program product of claim 15 wherein the computer program code for computing a representative metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries is configured to compute a single metadata entry as a regular expression.
17. The computer program product of claim 16 wherein the metadata entries for the plurality of files in the directory have a sequentially-ordered set of i-node numbers.
18. The computer program product of claim 15 wherein the computer program code for replacing values in the metadata entries is configured to replace timestamp values in the metadata entries for the plurality of files in the directory of the file system with a same approximated timestamp value.
19. The computer program product of claim 15 wherein the plurality of files in the directory of the file system are stored in a fast tier of storage of a storage system, and further comprising:
computer program code for storing the plurality of objects representing the plurality of files in the directory of the file system in a capacity tier of object storage of the storage system.
20. The computer program product of claim 19 further comprising:
computer program code for allocating a sequential range of i-nodes in the file system in the fast tier of the storage of the storage system; and
computer program code for restoring the plurality of files represented as the plurality of objects in the capacity tier of object storage of the storage system to the file system in the fast tier of storage of the storage system according to the common characteristics of the metadata entries for the plurality of files.
21. The computer program product of claim 20 wherein the computer program code for restoring the plurality of files represented as the plurality of objects in the capacity tier of object storage of the storage system to the file system in the fast tier of storage of the storage system according to the common characteristics of the metadata entries for the plurality of files comprises:
computer program code for computing the file names for the plurality of files according to the representative metadata entry and the common characteristic of the metadata entries; and
computer program code for assigning to each i-node in the sequential range of i-nodes an object from the capacity tier of object storage in the storage system according to its respective computed file name.