Metadata control in a load-balanced distributed storage system
A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. The metadata is grouped into buckets. Each bucket is stored in a group of computing devices. However, only the leader of the group is able to directly access a particular bucket at any given time.
1. A system comprising:
a plurality of storage devices, wherein a plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans more than one storage device of the plurality of storage devices; and
a plurality of computing devices communicatively coupled to each other and to the one or more storage devices via a network, wherein the plurality of computing devices is operable to maintain metadata that maps each failure resilient address space to one of the plurality of computing devices, and wherein the plurality of computing devices comprises a plurality of virtual file system (VFS) nodes, wherein the metadata is divided into a plurality of metadata shards, and wherein each metadata shard of the plurality of metadata shards is assigned to a unique group of VFS nodes of the plurality of VFS nodes.
2. The system of claim 1 , wherein each group of VFS nodes comprises five VFS nodes of the plurality of VFS nodes.
3. The system of claim 1 , wherein all VFS nodes in a group know and agree on a leader.
4. The system of claim 1 , wherein a particular one of the plurality of failure resilient address spaces may be accessed only by a leader of the group of VFS nodes to which a metadata shard associated with that particular failure resilient address space has been assigned.
5. The system of claim 1 , wherein each of the failure resilient address spaces is owned by only one of the plurality of computing devices at any given time.
6. The system of claim 1 , wherein each of the computing devices owns multiple of the failure resilient address spaces.
7. The system of claim 1 , wherein in the event of a failure of one of the plurality of computing devices, the plurality of computing devices are operable to redistribute metadata that was on the failed computing devices.
8. The system of claim 1 , wherein in the event of a change in the number of computing devices in the plurality of computing devices, the plurality of computing devices are operable to redistribute metadata according to a load value associated with each computing device in the plurality of computing devices.