Two level addressing in storage clusters
Digital objects are stored and accessed within a fixed content storage cluster by using a page mapping table and a pages index. A stream is read from the cluster by using a portion of its unique identifier as a key into the page mapping table. The page mapping table indicates a node holding a pages index indicating where the stream is stored. A stream is written to the cluster by storing the stream on any suitable node and then updating a pages index stored within the cluster. The cluster recovers from a node failure by first replicating streams from the failed node and reallocating a page mapping table to create a new pages index. The remaining nodes send records of the unique identifiers corresponding to objects they hold to the new pages index. A node is added to the cluster by reallocating a page mapping table.
1. A method of recovering from a node failure within a storage cluster, said method comprising:
detecting within said storage cluster that a first computer node out of a plurality of computer nodes is unavailable, said storage cluster including stored digital objects in persistent storage, each of said digital objects being identified by a unique identifier (UID), and wherein said unique identifiers having ranges of possible values;
accessing a table of said storage cluster, said table mapping each UID range to one of said computer nodes, and a first UID range being mapped to said first computer node within said table;
replacing said mapping of said first UID range to said first computer node with a mapping of said first UID range to a different computer node within said storage cluster;
causing each computer node within said storage cluster other than said first node to determine a set of unique identifiers falling within said first UID range indicating digital objects stored upon said each computer node;
receiving, at said different computer node from said each computer node, said sets that identify said unique identifiers and said each computer nodes where the digital objects corresponding to said unique identifiers are stored; and
updating an index on said different computer node to store mappings from said unique identifiers to said each computer node.
2. The method as recited in claim 1 wherein said storage cluster is a write-once read-many (WORM) fixed-content storage cluster.
3. The method as recited in claim 1 wherein said index maps a portion of said unique identifiers to said each computer node.
4. The method as recited in claim 1 wherein said index maps each unique identifier directly to a volume identifier identifying a disk within said storage cluster, said method further comprising:
mapping said volume identifier to said one of said computer nodes, whereby said index maps each unique identifier indirectly to said one of said computer nodes.
5. The method as recited in claim 1 wherein said table and said index are located in RAM.