Optimized layout in a two tier storage

1. A method comprising:

presenting, to an application, a file system, on a storage system comprising a fast tier of storage and a capacity tier of storage, having a global single namespace (SN) comprising a plurality of dynamically-loadable namespaces (DLNs) maintained in the capacity tier (CT) of object storage of the storage system;

in response to a job submitted by the application, dynamically loading a first dynamically-loadable namespace, including the metadata only for the first dynamically-loadable namespace, associated with the job from the capacity tier of object storage into the fast tier (FT) of storage of the storage system; and

periodically, by the data storage system, reducing metadata by:

determining common characteristics of metadata;

computing representative metadata entries according to the common characteristics; and

storing the representative metadata entries.

2. The method of claim 1 further comprising maintaining a map identifying associations between each dynamically-loadable namespace of the plurality of dynamically-loadable namespaces and an object identifier (OID) identifying the dynamically-loadable namespace in the capacity tier of object storage.

3. The method of claim 1 wherein dynamically loading a first dynamically-loaded namespace associated with the job from the capacity tier of object storage into a fast tier of storage comprises:

fetching the first dynamically-loadable namespace from the capacity tier of object storage;

receiving the first dynamically-loadable namespace from the capacity tier of object storage;

creating an active namespace in the fast tier of storage for the dynamically-loadable namespace; and

inserting the first dynamically-loadable namespace into the fast tier of storage.

4. The method of claim 1 further comprising opening a file for writing comprising:

receiving an open command from the application to write the file to the storage system;

performing a first lookup for the file in the fast tier;

receiving a first response that the first lookup failed;

performing a second lookup for the file in the capacity tier;

receiving a second response that the second lookup failed;

performing an inserting operation into the fast tier;

opening the file from the fast tier; and

acknowledging the open command from the application.

5. The method of claim 1 further comprising writing to a file comprising:

receiving a write command from the application;

performing an inserting operation into the fast tier;

writing the file to the fast tier; and

acknowledging the write command from the application.

6. The method of claim 1 further comprising flushing dirty data comprising:

receiving a flush command for dirty data from the application;

acknowledging the flush command prior to performing the flush command on the storage system;

performing a lookup for the dirty data in the fast tier;

receiving metadata from the fast tier identifying the dirty data;

creating an object in the capacity tier for the dirty data;

storing the dirty data to the object in the capacity tier; and

receiving an acknowledgement from the fast tier.

7. The method of claim 1 further comprising freeing capacity in the fast tier comprising:

inserting the dynamically-loaded namespace into the capacity tier as an object; and

deleting the dynamically-loaded namespace from the fast tier.

8. The method of claim 1 further comprising reading a file comprising:

receiving an open command from the application;

performing a first lookup for the file in the fast tier;

receiving a first response from the fast tier indicating success or failure of the first lookup;

if the first lookup succeeds because the file is resident in the fast tier in the first dynamically-loaded namespace loaded into the fast tier,

receiving metadata from the fast tier identifying the file; and

acknowledging the open command from the application;

if the first lookup fails because the file is not resident in the fast tier in the first dynamically-loaded namespace loaded into the fast tier,

performing a second lookup for the file in the capacity tier;

receiving a second response from the capacity tier indicating success or failure of the second lookup;

if the second lookup succeeds because the file is resident in the capacity tier,

retrieving the file from the capacity tier;

writing the file to the fast tier; and

acknowledging the open command from the application; and

if the second lookup fails because the file does not exist in either the fast tier or the capacity tier, failing the open command.

9. The method of claim 1 further comprising reading a file from the capacity tier comprising:

receiving a read command from the application;

performing a lookup for the file in the fast tier;

receiving metadata identifying the file in the capacity tier;

retrieving the file from the capacity tier;

loading the file into the fast tier;

reading the file from the fast tier; and

returning the file to the application.

10. The method of claim 1 further comprising freeing capacity in the fast tier comprising:

retrieving the dynamically-loaded namespace from the fast tier;

inserting the dynamically-loaded namespace into the capacity tier; and

deleting the dynamically-loaded namespace from the fast tier.

11. A storage system comprising:

a fast tier of storage;

a capacity tier of storage; and

computer program code for performing the operations of:

presenting, to an application, a file system, on the storage system, having a global single namespace (SN) comprising a plurality of dynamically-loadable namespaces (DLNs) maintained in the capacity tier (CT) of object storage of the storage system;

in response to a job submitted by the application, dynamically loading a first dynamically-loadable namespace, including the metadata only for the first dynamically-loadable namespace, associated with the job from the capacity tier of object storage into the fast tier (FT) of storage of the storage system; and

periodically, by the data storage system, reducing metadata by:

determining common characteristics of metadata;

computing representative metadata entries according to the common characteristics; and

storing the representative metadata entries.

12. The system of claim 11 further comprising maintaining a map identifying associations between each dynamically-loadable namespace of the plurality of dynamically-loadable namespaces and an object identifier (OID) identifying the dynamically-loadable namespace in the capacity tier of object storage.

13. The system of claim 11 wherein dynamically loading a first dynamically-loaded namespace associated with the job from the capacity tier of object storage into a fast tier of storage comprises:

fetching the first dynamically-loadable namespace from the capacity tier of object storage;

receiving the first dynamically-loadable namespace from the capacity tier of object storage;

creating an active namespace in the fast tier of storage for the dynamically-loadable namespace; and

inserting the first dynamically-loadable namespace into the fast tier of storage.

14. The system of claim 11 further comprising opening a file for writing comprising:

receiving an open command from the application to write the file to the storage system;

performing a first lookup for the file in the fast tier;

receiving a first response that the first lookup failed;

performing a second lookup for the file in the capacity tier;

receiving a second response that the second lookup failed;

performing an inserting operation into the fast tier;

opening the file from the fast tier; and

acknowledging the open command from the application.

15. The system of claim 11 further comprising writing to a file comprising:

receiving a write command from the application;

performing an inserting operation into the fast tier;

writing the file to the fast tier; and

acknowledging the write command from the application.

16. The system of claim 11 further comprising flushing dirty data comprising:

receiving a flush command for dirty data from the application;

acknowledging the flush command prior to performing the flush command on the storage system;

performing a lookup for the dirty data in the fast tier;

receiving metadata from the fast tier identifying the dirty data;

creating an object in the capacity tier for the dirty data;

storing the dirty data to the object in the capacity tier; and

receiving an acknowledgement from the fast tier.

17. The system of claim 11 further comprising freeing capacity in the fast tier comprising:

inserting the dynamically-loaded namespace into the capacity tier as an object; and

deleting the dynamically-loaded namespace from the fast tier.

18. The system of claim 11 further comprising reading a file comprising:

receiving an open command from the application;

performing a first lookup for the file in the fast tier;

receiving a first response from the fast tier indicating success or failure of the first lookup;

if the first lookup succeeds because the file is resident in the fast tier in the first dynamically-loaded namespace loaded into the fast tier,

receiving metadata from the fast tier identifying the file; and

acknowledging the open command from the application;

if the first lookup fails because the file is not resident in the fast tier in the first dynamically-loaded namespace loaded into the fast tier,

performing a second lookup for the file in the capacity tier;

receiving a second response from the capacity tier indicating success or failure of the second lookup;

if the second lookup succeeds because the file is resident in the capacity tier,

retrieving the file from the capacity tier;

writing the file to the fast tier; and

acknowledging the open command from the application; and

if the second lookup fails because the file does not exist in either the fast tier or the capacity tier, failing the open command.

19. The system of claim 11 further comprising reading a file from the capacity tier comprising:

receiving a read command from the application;

performing a lookup for the file in the fast tier;

receiving metadata identifying the file in the capacity tier;

retrieving the file from the capacity tier;

loading the file into the fast tier;

reading the file from the fast tier; and

returning the file to the application.

20. The system of claim 11 further comprising freeing capacity in the fast tier comprising:

retrieving the dynamically-loaded namespace from the fast tier;

inserting the dynamically-loaded namespace into the capacity tier; and

deleting the dynamically-loaded namespace from the fast tier.

21. A computer program product including a non-transitory computer readable storage medium having computer program code encoded thereon that when executed on a processor of a computer maintains dynamically loadable namespaces, the computer program code comprising:

computer program code for presenting, to an application, a file system, on a storage system, having a global single namespace (SN) comprising a plurality of dynamically-loadable namespaces (DLNs) maintained in a capacity tier (CT) of object storage of the storage system;

computer program code for, in response to a job submitted by the application, dynamically loading a first dynamically-loadable namespace, including the metadata only for the first dynamically-loadable namespace, associated with the job from the capacity tier of object storage into a fast tier (FT) of storage of the storage system; and

periodically, by the data storage system, reducing metadata by:

determining common characteristics of metadata;

computing representative metadata entries according to the common characteristics; and

storing the representative metadata entries.