Map-reduce ready distributed file system
A map-reduce compatible distributed file system that consists of successive component layers that each provide the basis on which the next layer is built provides transactional read-write-update semantics with file chunk replication and huge file-create rates. Containers provide the fundamental basis for data replication, relocation, and transactional updates. A container location database allows containers to be found among all file servers, as well as defining precedence among replicas of containers to organize transactional updates of container contents. Volumes facilitate control of data placement, creation of snapshots and mirrors, and retention of a variety of control and policy information. Also addressed is the use of distributed transactions in a map-reduce system; the use of local and distributed snapshots; replication, including techniques for reconciling the divergence of replicated data after a crash; and mirroring.
1. A non-transitory storage medium comprising instructions that upon execution cause a system to:
replicate a container to store a plurality of replica containers among a plurality of cluster nodes;
maintain, in a container location database (CLDB), information about where each of the plurality of replica containers is located in the plurality of cluster nodes;
store, in the CLDB, an epoch number for the container, the epoch number referring to an epoch;
designate a replica container of the plurality of replica containers as a master container, wherein the master container is to receive an update to the container and propagate the update to other replica containers of the plurality of replica containers;
update the epoch number in response to a change of a replication structure for the container, the change of the replication structure based on a change in an arrangement of the plurality of cluster nodes; and
record, in a transaction history and in association with the epoch, the change of the replication structure for the container.
2. The non-transitory storage medium of claim 1 , wherein the change in the arrangement of the plurality of cluster nodes comprises an addition of a given cluster node to the plurality of cluster nodes.
3. The non-transitory storage medium of claim 2 , wherein the given cluster node added to the plurality of cluster nodes includes an out-of-date replica container.
4. The non-transitory storage medium of claim 3 , wherein the instructions upon execution cause the system to:
contact, by the given cluster node including the out-of-date replica container, the CLDB; and
decide, by the CLDB, an action to take in response to the contact by the given cluster node.
5. The non-transitory storage medium of claim 4 , wherein the deciding by the CLDB comprises:
determining that a sufficient quantity of replica containers is present in the plurality of cluster nodes, and
instructing, by the CLDB, the given cluster node to discard the out-of-date replica container.
6. The non-transitory storage medium of claim 4 , wherein the deciding by the CLDB comprises:
assigning, by the CLDB, the out-of-date replica container to the replication structure, and
initiating, by the CLDB, a synchronization process to bring the out-of-date replica container up to a current state.
7. The non-transitory storage medium of claim 1 , wherein the change in the arrangement of the plurality of cluster nodes comprises a failure of a given cluster node that includes a replica container of the plurality of replica containers.
8. The non-transitory storage medium of claim 7 , wherein the instructions upon execution cause the system to:
detect that the given cluster node includes the master container, and
in response to detecting that the given cluster node includes the master container, designate another replica container in another cluster node of the plurality of cluster nodes as the master container.
9. The non-transitory storage medium of claim 7 , wherein the instructions upon execution cause the system to:
detect that the given cluster node does not include the master container,
add another cluster node to the plurality of cluster nodes to replace the given cluster node, and store a new replica container at the another cluster node.
10. The non-transitory storage medium of claim 1 , wherein the change in the arrangement of the plurality of cluster nodes comprises the plurality of cluster nodes starting, and wherein the instructions upon execution cause the system to:
until a minimum quantity of the plurality of cluster nodes have started, disallow any updates to the container.
11. The non-transitory storage medium of claim 10 , wherein the instructions upon execution cause the system to:
in response to detecting that the minimum quantity of the plurality of cluster nodes have started, select a cluster node of the minimum quantity of the plurality of cluster nodes to include the master container.
12. The non-transitory storage medium of claim 11 , wherein the selected cluster node is a cluster node that has seen a most recent epoch.
13. A system comprising:
a processor; and
a non-transitory storage medium storing instructions executable on the processor to:
replicate a container to store a plurality of replica containers among a plurality of cluster nodes;
maintain, in a container location database (CLDB), information about where each of the plurality of replica containers is located in the plurality of cluster nodes;
store, in the CLDB, an epoch number for the container, the epoch number referring to an epoch;
designate a replica container of the plurality of replica containers as a master container, wherein the master container is to receive an update to the container and propagate the update to other replica containers of the plurality of replica containers;
update the epoch number in response to a change of a replication structure for the container, the change of the replication structure based on a change in an arrangement of the plurality of cluster nodes; and
record, in a transaction history and in association with the epoch, the change of the replication structure for the container.
14. The system of claim 13 , wherein the change in the arrangement of the plurality of cluster nodes comprises an addition of a given cluster node to the plurality of cluster nodes, wherein the given cluster node added to the plurality of cluster nodes includes an out-of-date replica container, and wherein the instructions are executable on the processor to:
contact, by the given cluster node including the out-of-date replica container, the CLDB; and
decide, by the CLDB, an action to take in response to the contact by the given cluster node.
15. The system of claim 13 , wherein the change in the arrangement of the plurality of cluster nodes comprises a failure of a given cluster node that includes a replica container of the plurality of replica containers, and wherein the instructions are executable on the processor to:
detect that the given cluster node includes the master container, and
in response to detecting that the given cluster node includes the master container, designate another replica container in another cluster node of the plurality of cluster nodes as the master container.
16. The system of claim 13 , wherein the change in the arrangement of the plurality of cluster nodes comprises the plurality of cluster nodes starting, and wherein the instructions are executable on the processor to:
until a minimum quantity of the plurality of cluster nodes have started, disallow any updates to the container.
17. A method of a system, the method comprising:
replicating a container to store a plurality of replica containers among a plurality of cluster nodes;
maintaining, in a container location database (CLDB), information about where each of the plurality of replica containers is located in the plurality of cluster nodes;
storing, in the CLDB, an epoch number for the container, the epoch number referring to an epoch;
designating a replica container of the plurality of replica containers as a master container, wherein the master container is to receive an update to the container and propagate the update to other replica containers of the plurality of replica containers;
updating the epoch number in response to a change of a replication structure for the container, the change of the replication structure based on a change in an arrangement of the plurality of cluster nodes; and
recording, in a transaction history and in association with the epoch, the change of the replication structure for the container.
18. The method of claim 17 , wherein the change in the arrangement of the plurality of cluster nodes comprises an addition of a given cluster node to the plurality of cluster nodes.
19. The method of claim 17 , wherein the change in the arrangement of the plurality of cluster nodes comprises a failure of a given cluster node that includes a replica container of the plurality of replica containers.
20. The method of claim 17 , wherein the change in the arrangement of the plurality of cluster nodes comprises the plurality of cluster nodes starting, and wherein the method comprises:
until a minimum quantity of the plurality of cluster nodes have started, disallowing any updates to the container; and
in response to detecting that the minimum quantity of the plurality of cluster nodes have started, selecting a cluster node of the minimum quantity of the plurality of cluster nodes to include the master container;
wherein the selected cluster node is a cluster node that has seen a most recent epoch.