NETWORK SYSTEM TO DISTRIBUTE CHUNKS ACROSS MULTIPLE PHYSICAL NODES WITH DISK SUPPORT FOR OBJECT STORAGE
A method of storing a file in a storage system that includes a plurality of memory-storage hosts includes: specifying a unique chunk identifier for a memory chunk included in the file; specifying a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host; providing metadata corresponding to the hash mapping to the memory-storage hosts; and storing the memory chuck at the one or more storage locations identified by the hash mapping by providing the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.
1 . A method of storing a file in a storage system that includes a plurality of memory-storage hosts, the method comprising:
specifying a unique chunk identifier for a memory chunk included in the file;
specifying a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host;
providing metadata corresponding to the hash mapping to the memory-storage hosts; and
storing the memory chuck at the one or more storage locations identified by the hash mapping by providing the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.
2 . The method of claim 1 , wherein each memory-storage host maintains a cache memory including values for the metadata and the hash mapping.
3 . The method of claim 1 , wherein each memory-storage host implements the hash mapping at an interface of said each memory-storage host.
4 . The method of claim 1 , wherein each memory storage host includes:
a disk storage unit that is accessible by a addressing a block size and a block offset, and
an interface unit that implements the hash mapping to relate the unique chunk identifier to a corresponding block size and block offset.
5 . The method of claim 1 , wherein the unique chunk identifier includes a node ID (identification number) to identify a memory-storage host, a LUN ID to identify a Logical Unit Number (LUN) within the memory-storage host, and a Chunk ID to identify a chunk of memory with the LUN.
6 . The method of claim 1 , wherein the file includes a plurality of memory chunks that have a uniform size given by an integral multiple of a memory block size associated with the storage system.
7 . The method of claim 1 , further comprising:
updating the metadata when a memory-storage host is added to the system or deleted from the system; and
providing the updated metadata to the memory-storage hosts.
8 . A non-transitory computer-readable medium that stores a computer program for storing a file in a storage system that includes a plurality of memory-storage hosts, the computer program including instructions that, when executed by at least one computer, cause the at least one computer to perform operations comprising:
specifying a unique chunk identifier for a memory chunk included in the file;
specifying a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host;
providing metadata corresponding to the hash mapping to the memory-storage hosts; and
storing the memory chuck at the one or more storage locations identified by the hash mapping by providing the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.
9 . The non-transitory computer-readable medium of claim 8 , wherein each memory-storage host maintains a cache memory including values for the metadata and the hash mapping.
10 . The non-transitory computer-readable medium of claim 8 , wherein each memory-storage host implements the hash mapping at an interface of said each memory-storage host.
11 . The non-transitory computer-readable medium of claim 8 , wherein each memory storage host includes:
a disk storage unit that is accessible by a addressing a block size and a block offset, and
an interface unit that implements the hash mapping to relate the unique chunk identifier to a corresponding block size and block offset.
12 . The non-transitory computer-readable medium of claim 8 , wherein the unique chunk identifier includes a node ID (identification number) to identify a memory-storage host, a LUN ID to identify a Logical Unit Number (LUN) within the memory-storage host, and a Chunk ID to identify a chunk of memory with the LUN.
13 . The non-transitory computer-readable medium of claim 8 , wherein the file includes a plurality of memory chunks that have a uniform size given by an integral multiple of a memory block size associated with the storage system.
14 . The non-transitory computer-readable medium of claim 8 , wherein the computer program further includes instructions that, when executed by the at least one computer, cause the at least one computer to perform operations comprising:
updating the metadata when a memory-storage host is added to the system or deleted from the system; and
providing the updated metadata to the memory-storage hosts.
15 . A system storing a file in a storage system that includes a plurality of memory-storage hosts, the system including at least one computer configured to perform operations for computer-implemented modules including:
a chunk-identifier module that specifies a unique chunk identifier for a memory chunk included in the file;
a hash-mapping module that specifies a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host;
a storage module that provides metadata corresponding to the hash mapping to the memory-storage hosts and further provides the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.
16 . The system of claim 15 , wherein each memory-storage host maintains a cache memory including values for the metadata and the hash mapping.
17 . The system of claim 15 , wherein each memory-storage host implements the hash mapping at an interface of said each memory-storage host.
18 . The system of claim 15 , wherein each memory storage host includes:
a disk storage unit that is accessible by a addressing a block size and a block offset, and
an interface unit that implements the hash mapping to relate the unique chunk identifier to a corresponding block size and block offset.
19 . The system of claim 15 , wherein the unique chunk identifier includes a node ID (identification number) to identify a memory-storage host, a LUN ID to identify a Logical Unit Number (LUN) within the memory-storage host, and a Chunk ID to identify a chunk of memory with the LUN.
20 . The system of claim 15 , wherein the file includes a plurality of memory chunks that have a uniform size given by an integral multiple of a memory block size associated with the storage system.