Methods and apparatus for filtering dynamically loadable namespaces (DLNs)
Methods and apparatus are provided for filtering dynamically loadable namespaces (DLNs). An exemplary method comprises, in response to a job submitted by an application, obtaining a DLN portion of a global single namespace of a file system, wherein the DLN is associated with the job and is maintained in a capacity tier of a storage system; obtaining filtering directives from a user; reducing the DLN using a filtering mechanism on a directory tree associated with the DLN, based on the filtering directives, by removing files in the directory tree of the DLN that do not satisfy requirements of the filtering directives to generate a filtered DLN; and dynamically loading the filtered DLN, including reduced metadata for the filtered DLN relative to the DLN, from the capacity tier into a performance tier of the storage system for processing by the application.
1. A method comprising:
in response to a job submitted by an application, obtaining, using a two-tier daemon, a dynamically loadable namespace (DLN) portion of a global single namespace of a file system, wherein said DLN is associated with the job and is obtained by said two-tier daemon from a capacity tier of object storage of a storage system;
obtaining one or more filtering directives from a user;
reducing said DLN using a filtering mechanism on a directory tree associated with said DLN, based on said one or more filtering directives, by removing one or more files in said directory tree of said DLN that do not satisfy one or more requirements of said one or more filtering directives to generate a filtered DLN; and
dynamically loading, by said two-tier daemon, the filtered DLN, including reduced metadata for the filtered DLN relative to said DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by said application, wherein said dynamically loading by said two-tier daemon comprises:
creating an entry into a performance tier namespace of said performance tier;
receiving a first acknowledgement from said performance tier;
inserting said filtered DLN into said performance tier namespace of said performance tier by populating said filtered DLN with one or more objects representing one or more files of said filtered DLN; and
receiving a second acknowledgement from said performance tier,
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2. The method of claim 1 , wherein the reducing is performed by one or more of said two-tier daemon in the performance tier and a job scheduler, and wherein said two-tier daemon provides a third acknowledgement to said job scheduler in response to said receiving said second acknowledgement from said performance tier.
3. The method of claim 1 , further comprising the step of merging said filtered DLN into said DLN following one or more modifications to said filtered DLN.
4. The method of claim 3 , wherein said merging comprises the steps of converting a content of said performance tier into a merged DLN, traversing a directory tree associated with said DLN and, for each entry in said DLN, inserting a given entry from said merged DLN when said given entry is not in said DLN.
5. The method of claim 3 , wherein said merging does not insert deleted files in said merged DLN.
6. The method of claim 1 , further comprising the step of reorganizing one or more files in said filtered DLN based on one or more reordering directives.
7. The method of claim 1 , wherein the dynamically loading the filtered DLN into the performance tier comprises:
fetching the DLN from the capacity tier of object storage;
receiving the DLN from the capacity tier of object storage;
performing said reducing of said DLN based on said one or more filtering directives to generate said filtered DLN;
creating a namespace in the performance tier for the filtered DLN; and
inserting the filtered DLN into the performance tier of storage.
8. A non-transitory machine-readable recordable storage medium, wherein one or more software programs when executed by one or more processing devices implement the following steps:
in response to a job submitted by an application, obtaining, using a two-tier daemon, a dynamically loadable namespace (DLN) portion of a global single namespace of a file system, wherein said DLN is associated with the job and is obtained by said two-tier daemon from a capacity tier of object storage of a storage system;
obtaining one or more filtering directives from a user;
reducing said DLN using a filtering mechanism on a directory tree associated with said DLN, based on said one or more filtering directives, by removing one or more files in said directory tree of said DLN that do not satisfy one or more requirements of said one or more filtering directives to generate a filtered DLN; and
dynamically loading, by said two-tier daemon, the filtered DLN, including reduced metadata for the filtered DLN relative to said DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by said application, wherein said dynamically loading by said two-tier daemon comprises:
creating an entry into a performance tier namespace of said performance tier;
receiving a first acknowledgement from said performance tier;
inserting said filtered DLN into said performance tier namespace of said performance tier by populating said filtered DLN with one or more objects representing one or more files of said filtered DLN; and
receiving a second acknowledgement from said performance tier.
9. The non-transitory machine-readable recordable storage medium of claim 8 , further comprising the step of merging said filtered DLN into said DLN following one or more modifications to said filtered DLN.
10. The non-transitory machine-readable recordable storage medium of claim 9 , wherein said merging comprises the steps of converting a content of said performance tier into a merged DLN, traversing a directory tree associated with said DLN and, for each entry in said DLN, inserting a given entry from said merged DLN when said given entry is not in said DLN.
11. The non-transitory machine-readable recordable storage medium of claim 9 , wherein said merging does not insert deleted files in said merged DLN.
12. The non-transitory machine-readable recordable storage medium of claim 8 , further comprising the step of reorganizing one or more files in said filtered DLN based on one or more reordering directives.
13. The non-transitory machine-readable recordable storage medium of claim 8 , wherein the reducing is performed by one or more of said two-tier daemon in the performance tier and a job scheduler, and wherein said two-tier daemon provides a third acknowledgement to said job scheduler in response to said receiving said second acknowledgement from said performance tier.
14. A system, comprising:
a memory; and
at least one processing device comprising a processor coupled to the memory, the at least one processing device being configured to:
in response to a job submitted by an application, obtaining, using a two-tier daemon, a dynamically loadable namespace (DLN) portion of a global single namespace of a file system, wherein said DLN is associated with the job and is obtained by said two-tier daemon from a capacity tier of object storage of a storage system;
obtain one or more filtering directives from a user;
reduce said DLN using a filtering mechanism on a directory tree associated with said DLN, based on said one or more filtering directives, by removing one or more files in said directory tree of said DLN that do not satisfy one or more requirements of said one or more filtering directives to generate a filtered DLN; and
dynamically load, by said two-tier daemon, the filtered DLN, including reduced metadata for the filtered DLN relative to said DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by said application, wherein said dynamically loading by said two-tier daemon comprises:
creating an entry into a performance tier namespace of said performance tier;
receiving a first acknowledgement from said performance tier;
inserting said filtered DLN into said performance tier namespace of said performance tier by populating said filtered DLN with one or more objects representing one or more files of said filtered DLN; and
receiving a second acknowledgement from said performance tier.
15. The system of claim 14 , wherein the reducing is performed by one or more of said two-tier daemon in the performance tier and a job scheduler, and wherein said two-tier daemon provides a third acknowledgement to said job scheduler in response to said receiving said second acknowledgement from said performance tier.
16. The system of claim 14 , wherein said at least one processing device is further configured to merge said filtered DLN into said DLN following one or more modifications to said filtered DLN.
17. The system of claim 16 , wherein said merging comprises the steps of converting a content of said performance tier into a merged DLN, traversing a directory tree associated with said DLN and, for each entry in said DLN, inserting a given entry from said merged DLN when said given entry is not in said DLN.
18. The system of claim 16 , wherein said merging does not insert deleted files in said merged DLN.
19. The system of claim 14 , wherein said at least one processing device is further configured to reorganize one or more files in said filtered DLN based on one or more reordering directives.
20. The system of claim 14 , further comprising a performance tier of storage and a capacity tier of object storage.