Method and system for global namespace with consistent hashing

1. A method for writing data to a storage pool, comprising:

receiving a virtual identifier (ID) and an offset for an object;

extracting a node identifier (ID) that identifies a first storage server in the storage pool from the virtual ID;

obtaining an object layout (OL) for the object from the first storage server;

hashing an object ID and an offset ID to obtain a hashed value, wherein the virtual ID comprises the object ID and wherein the offset ID is generated from the offset;

identifying a second storage server in the storage pool using a global namespace layout (GL), the OL, and the hashed value; and

issuing a write request to write data to the object in the second storage server, wherein the write request comprises the object ID and the offset.

2. The method of claim 1 , wherein identifying the second storage server comprises using consistent hashing.

3. The method of claim 2 , wherein the storage pool comprises a plurality of storage servers, wherein the GL specifies which of the plurality of storage servers were writeable for each GL sequence number and how writes were distributed across the plurality of storage servers writeable for each GL sequence.

4. The method of claim 3 ,

wherein the OL specifies an OL sequence number, wherein the OL sequence number is set when the object is created, wherein the OL sequence number corresponds to a GL sequence number when the object was created, and

wherein identifying the second storage server in the storage pool comprises:

identifying a GL sequence number from the GL using the OL sequence number;

determining, using the GL, which of the plurality of storage servers was writeable at the GL sequence number to obtain a set of storage servers, and

identifying the second storage server from the set of storage servers.

5. The method of claim 4 , wherein identifying the second storage server from the set of storage servers comprising using consistent hashing and information from the GL about the how writes were distributed across the set of storage servers.

6. The method of claim 1 , wherein obtaining the OL from the first storage server comprises:

hashing the virtual ID and a special offset to obtain a second hashed value, and

obtaining the OL from the first storage server using the second hashed value.

7. The method of claim 1 , wherein the OL comprises an OL sequence number, the virtual ID, and a stripe width.

8. The method of claim 1 , wherein the GL comprises:

a first server entry for the first storage server comprising:

the node ID identifying the first storage server,

a first node sequence number corresponding to a GL sequence number when the first storage server was added to the storage pool, and

a node point used to determine which write requests are serviced by the first storage server.

9. The method of claim 1 , further comprising:

determining that the second storage server does not include a copy of the OL;

issuing a request to store the OL to the second storage server, wherein the request to store the OL specifies the virtual ID and a special offset.

10. The method of claim 1 , further comprising:

receiving a second virtual ID and a second offset ID for a second object;

extracting a second node ID that identifies a third storage server in the storage pool from the second virtual ID;

obtaining a second OL for the second object from the third storage server;

hashing a second object ID and a second offset ID to obtain a second hashed value, wherein the second virtual ID comprises the second object ID, and wherein the second offset ID is generated from the second offset;

identifying the second storage server in the storage pool using the GL, the second OL, and the second hashed value; and

issuing a second write request to write data to the second object in the second storage server, wherein the second write request comprises the second object ID and the second offset.

11. The method of claim 1 , further comprising:

receiving a second logical address for a second object;

extracting a second node ID that identifies the second storage server in the storage pool from the second virtual ID;

obtaining a second OL for the second object from the second storage server;

hashing a second object ID and a second offset ID to obtain a second hashed value, wherein the second virtual ID comprises the second object ID, and wherein the second offset ID is generated from the second offset;

identifying the first storage server in the storage pool using the GL, the second OL, and the second hashed value; and

issuing a second write request to write data to the second object in the first storage server, wherein the second write request comprises the second object ID and the second offset.

12. A method for reading data to a storage pool, comprising:

receiving a virtual ID and an offset for an object;

extracting a node identifier (ID) that identifies a first storage server in the storage pool from the virtual ID;

obtaining an object layout (OL) for the object from the first storage server;

hashing an object ID and an offset ID to obtain a hashed value, wherein the virtual ID comprises the object ID and wherein the offset ID is generated from the offset;

identifying, using consistent hashing, a second storage server in the storage pool using a global namespace layout (GL), the OL, and the hashed value; and

issuing a read request to read data from the object in the second storage server, wherein the read request comprises the object ID and the offset.

13. The method of claim 12 , wherein the identifying the second storage server comprises:

identifying a set of storage servers from the plurality of storage servers in the storage pool using the GL and an OL sequence number from the OL;

identifying the second storage server from the set of storage servers using the hashed value and consistent hashing.

14. The method of claim 12 , wherein the offset ID is further generated using a stripe width, wherein the OL comprises the stripe width.

15. A method for writing data to a storage pool, comprising:

receiving a request for a new object;

selecting a storage server from the storage pool, wherein the storage server is identified using a Node ID;

requesting the storage server to generate a virtual ID, wherein the virtual ID comprises the node ID and an object ID for the object;

generating an object layout (OL) for the object using the virtual ID;

storing the OL in the storage server;

receiving a write request to write data to the object, wherein the write request comprises an offset;

generating an offset ID from the offset;

hashing the object ID and the offset ID to obtain a hashed value;

identifying a second storage server in the storage pool using a global namespace layout (GL), the OL, and the hashed value; and

issuing a write request to write the data to the object in the second storage server, wherein the write request comprises the object ID and the offset.

16. The method of claim 15 , wherein selecting the storage server from the storage pool comprises using at least one selected from a group consisting of randomly selecting the storage server and selecting the storage server using a round robin algorithm.

17. The method of claim 15 , wherein the node ID uniquely identifies the storage server in the storage pool.

18. The method of claim 15 , wherein storing the OL in the storage server comprises:

hashing the virtual ID and a special offset to obtain a second hashed value; and

storing the OL in the storage server using the second hashed value.

19. The method of claim 15 , wherein generating the OL further comprises using a global sequence number from the GL.