Map-reduce ready distributed file system
A map-reduce compatible distributed file system that consists of successive component layers that each provide the basis on which the next layer is built provides transactional read-write-update semantics with file chunk replication and huge file-create rates. Containers provide the fundamental basis for data replication, relocation, and transactional updates. A container location database allows containers to be found among all file servers, as well as defining precedence among replicas of containers to organize transactional updates of container contents. Volumes facilitate control of data placement, creation of snapshots and mirrors, and retention of a variety of control and policy information. Also addressed is the use of distributed transactions in a map-reduce system; the use of local and distributed snapshots; replication, including techniques for reconciling the divergence of replicated data after a crash; and mirroring.
1. A method of operating a distributed file system comprising:
maintaining at least one inode in a container on a node of a plurality of cluster nodes, wherein the at least one inode includes attributes including an object type, direct data pointers to disk blocks in the container, and an indirect data pointer, wherein the indirect data pointer points to a structure and the structure points to indirect data;
replicating the container to store replica containers among the plurality of cluster nodes;
designating a master container from among the replica containers, wherein updates to the container are sent to the master container, and the master container propagates the updates to others of the replica containers; and
maintaining a container location database (CLDB), wherein the CLDB designates an ordering of a replication chain comprising the replica containers and keeps information about locations of the replica containers stored among the plurality of cluster nodes.
2. The method of claim 1 , in response to a read data request including a file identifier comprising a reference to the at least one inode and a container identifier:
determining one or more cluster nodes to be a location of one or more of the replica containers;
reading an inode at a cluster node picked from the determined one or more cluster nodes, using the read data request and the reference to the at least one inode; and
returning data read via direct data pointers or an indirect data pointer of the inode read at the cluster node.
3. The method of claim 1 , comprising representing a local file using the direct data pointers, wherein the direct data pointers contain the local file entirely within disk blocks of the container, and wherein the object type indicates local file type.
4. The method of claim 1 , wherein the object type indicates local file type, and the structure pointed to by the indirect data pointer is a B-tree.
5. The method of claim 1 , comprising representing a chunked file by using a file identifier map for the structure pointed to by the indirect data pointer, the file identifier map including an array of file identifiers associated with a plurality of file chunks stored in other containers in the plurality of cluster nodes, and wherein the object type indicates chunked file type.
6. The method of claim 5 , comprising:
storing a size of each of the plurality of file chunks within the attributes of the at least one inode, wherein the array of file identifiers indicates locations of the file chunks.
7. The method of claim 1 , maintaining the CLDB in inodes in containers on a plurality of redundant servers.
8. The method of claim 1 , wherein the CLDB is to keep information about versions of the replica containers.
9. The method of claim 1 , wherein the master container propagates the updates to others of the replica containers in a linear replication pattern.
10. The method of claim 1 , wherein the master container propagates the updates to others of the replica containers in a star replication pattern.
11. A computer system comprising:
a processor; and
a non-transitory storage medium storing instructions that, when executed, cause the processor to:
maintain at least one inode in a container on a node of a plurality of cluster nodes, wherein the at least one inode includes attributes including an object type, direct data pointers to disk blocks in the container, and an indirect data pointer, wherein the indirect data pointer points to a structure and the structure points to indirect data,
replicate the container to store replica containers among the plurality of cluster nodes,
designate a master container from among the replica containers, wherein updates to the container are sent to the master container, and the master container propagates the updates to others of the replica containers, and
maintain a container location database (CLDB), wherein the CLDB designates an ordering of a replication chain comprising the replica containers and keeps information about locations of the replica containers stored among the plurality of cluster nodes.
12. The computer system of claim 11 , the instructions further causing the processor to respond to a read data request including a file identifier comprising a reference to the at least one inode and a container identifier by:
determining one or more cluster nodes to be a location of one or more of the replica containers,
reading an inode at a cluster node picked from the determined one or more cluster nodes, using the read data request and the reference to the at least one inode, and
returning data read via direct data pointers or an indirect data pointer of the inode read at the cluster node.
13. The computer system of claim 11 , wherein the object type indicates local file type, and the container uses the direct data pointers to represent a local file contained entirely within the disk blocks in the container.
14. The computer system of claim 11 , wherein the object type indicates local file type, and the structure pointed to by the indirect data pointer is a B-tree.
15. The computer system of claim 11 , wherein the object type indicates chunked file type, and the container represents a chunked file by using a file identifier map for the structure pointed to by the indirect data pointer, the file identifier map including an array of file identifiers associated with a plurality of file chunks stored in other containers in the plurality of cluster nodes.
16. The computer system of claim 15 , wherein the attributes include a size of each of the plurality of file chunks and the array of file identifiers indicates locations of the file chunks.
17. The computer system of claim 11 , the instructions further causing the processor to maintain the CLDB in inodes in containers on a plurality of redundant servers.
18. The computer system of claim 17 , wherein the CLDB is replicated to CLDB nodes of the redundant servers and a CLDB master is elected from among the CLDB nodes by a coordination service.
19. The computer system of claim 11 , wherein the CLDB keeps information about versions of the replica containers.
20. The computer system of claim 11 , wherein the master container propagates the updates to others of the replica containers in a star replication pattern or a linear replication pattern.