Methods and apparatus for filtering dynamically loadable namespaces (DLNs)
View Patent ↗Methods and apparatus are provided for filtering dynamically loadable namespaces (DLNs). In one exemplary embodiment, a method comprises the steps of, in response to a job submitted by an application, obtaining a dynamically loadable namespace (DLN) portion of a global single namespace of a file system, wherein the DLN is associated with the job and is maintained in a capacity tier of object storage of a storage system; obtaining one or more filtering directives from a user; reducing the DLN based on the one or more filtering directives to generate a filtered DLN; and dynamically loading the filtered DLN, including the metadata only for the filtered DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by the application. The filtered DLN is merged into the original DLN following one or more modifications to the filtered DLN.
1. A method comprising:
in response to a job submitted by an application, obtaining a dynamically loadable namespace (DLN) portion of a global single namespace of a file system, wherein said DLN is associated with the job and is maintained in a capacity tier of object storage of a storage system;
obtaining one or more filtering directives from a user;
reducing, using at least one processing device, said DLN based on said one or more filtering directives by removing one or more files from said DLN that do not satisfy one or more requirements of said one or more filtering directives to generate a filtered DLN, wherein the reducing comprises traversing a directory tree associated with said DLN and, for each entry in said DLN, creating an entry in a directory tree associated with said filtered DLN when said entry satisfies said one or more requirements of said one or more filtering directives; and
dynamically loading, using said at least one processing device, the filtered DLN, including the metadata only for the filtered DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by said application.
2. The method of claim 1 , wherein the reducing is performed by one or more of a daemon in the performance tier and a job scheduler.
3. The method of claim 1 , further comprising the step of merging said filtered DLN into said DLN following one or more modifications to said filtered DLN.
4. The method of claim 3 , wherein said merging comprises the steps of converting a content of said performance tier into a merged DLN, traversing a directory tree associated with said DLN and, for each entry in said DLN, and inserting a given entry from said merged DLN when said given entry is not in said DLN.
5. The method of claim 3 , wherein said merging does not insert deleted files in said merged DLN.
6. The method of claim 1 , further comprising the step of reorganizing one or more files in said filtered DLN based on one or more reordering directives.
7. The method of claim 1 , wherein the dynamically loading the filtered DLN into the performance tier comprises:
fetching the DLN from the capacity tier of object storage;
receiving the DLN from the capacity tier of object storage;
performing said reducing of said DLN based on said one or more filtering directives to generate said filtered DLN;
creating an active namespace in the performance tier for the filtered DLN; and
inserting the filtered DLN into the performance tier of storage.
8. The method of claim 1 , further comprising reading a file comprising:
receiving an open command from the application;
performing a first lookup for the file in the performance tier;
receiving a first response from the performance tier indicating success or failure of the first lookup;
when the first lookup succeeds because the file is resident in the performance tier in the filtered DLN loaded into the performance tier,
receiving metadata from the performance tier identifying the file; and
acknowledging the open command from the application.
9. A non-transitory machine-readable recordable storage medium, wherein one or more software programs when executed by one or more processing devices implement the following steps:
in response to a job submitted by an application, obtaining a dynamically loadable namespace (DLN) portion of a global single namespace of a file system, wherein said DLN is associated with the job and is maintained in a capacity tier of object storage of a storage system;
obtaining one or more filtering directives from a user;
reducing, using at least one processing device, said DLN based on said one or more filtering directives by removing one or more files from said DLN that do not satisfy one or more requirements of said one or more filtering directives to generate a filtered DLN, wherein the reducing comprises traversing a directory tree associated with said DLN and, for each entry in said DLN, creating an entry in a directory tree associated with said filtered DLN when said entry satisfies said one or more requirements of said one or more filtering directives; and
dynamically loading, using said at least one processing device, the filtered DLN, including the metadata only for the filtered DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by said application.
10. The non-transitory machine-readable recordable storage medium of claim 9 , further comprising the step of merging said filtered DLN into said DLN following one or more modifications to said filtered DLN.
11. The non-transitory machine-readable recordable storage medium of claim 10 , wherein said merging comprises the steps of converting a content of said performance tier into a merged DLN, traversing a directory tree associated with said DLN and, for each entry in said DLN, and inserting a given entry from said merged DLN when said given entry is not in said DLN.
12. The non-transitory machine-readable recordable storage medium of claim 10 , wherein said merging does not insert deleted files in said merged DLN.
13. The non-transitory machine-readable recordable storage medium of claim 9 , further comprising the step of reorganizing one or more files in said filtered DLN based on one or more reordering directives.
14. A system, comprising:
a memory; and
at least one processing device configured to:
in response to a job submitted by an application, obtain a dynamically loadable namespace (DLN) portion of a global single namespace of a file system, wherein said DLN is associated with the job and is maintained in a capacity tier of object storage of a storage system;
obtain one or more filtering directives from a user;
reduce, using at least one processing device, said DLN based on said one or more filtering directives by removing one or more files from said DLN that do not satisfy one or more requirements of said one or more filtering directives to generate a filtered DLN, wherein the reducing comprises traversing a directory tree associated with said DLN and, for each entry in said DLN, creating an entry in a directory tree associated with said filtered DLN when said entry satisfies said one or more requirements of said one or more filtering directives; and
dynamically load, using said at least one processing device, the filtered DLN, including the metadata only for the filtered DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by said application.
15. The system of claim 14 , wherein the reducing is performed by one or more of a daemon in the performance tier and a job scheduler.
16. The system of claim 14 , wherein said at least one processing device is further configured to merge said filtered DLN into said DLN following one or more modifications to said filtered DLN.
17. The system of claim 16 , wherein said merging comprises the steps of converting a content of said performance tier into a merged DLN, traversing a directory tree associated with said DLN and, for each entry in said DLN, and inserting a given entry from said merged DLN when said given entry is not in said DLN.
18. The system of claim 16 , wherein said merging does not insert deleted files in said merged DLN.
19. The system of claim 14 , wherein said at least one processing device is further configured to reorganize one or more files in said filtered DLN based on one or more reordering directives.
20. The system of claim 14 , further comprising a performance tier of storage and a capacity tier of object storage.