Metadata control in a load-balanced distributed storage system
A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. The metadata is grouped into buckets. Each bucket is stored in a group of computing devices. However, only the leader of the group is able to directly access a particular bucket at any given time.
1. A method, comprising:
for a bucket of a plurality of buckets:
associating the bucket with a unique group of computing devices;
selecting a particular computing device, of the unique group of computing devices, as a leader of the bucket associated with the unique group; and
allowing access to a particular address space only by the leader of the bucket to which metadata associated with that particular address space has been distributed.
2. The method of claim 1 , wherein the unique group of computing devices comprises a plurality of virtual file system (VFS) nodes.
3. The method of claim 1 , wherein each unique group of computing devices comprises five VFS nodes of a plurality of VFS nodes.
4. The method of claim 1 , wherein all VFS nodes in a unique group know and agree on the leader of the bucket.
5. The method of claim 1 , wherein each address space has only one leader at any given time.
6. The method of claim 1 , wherein the particular computing device is a leader of multiple buckets.
7. The method of claim 1 , wherein the method comprises, in the event of a failure of one of the unique group of computing devices, redistributing metadata that was on the failed computing device.
8. The method of claim 1 , wherein the method comprises, in the event of a change in a number of computing devices in the unique group of computing devices, redistributing metadata according to a load value associated with each computing device in the plurality of computing devices.
9. The method of claim 1 , wherein the method comprises changing group leadership in the event of a load imbalance.
10. The method of claim 1 , wherein the method comprises splitting and redistributing buckets of the plurality of buckets in the event of a load imbalance.
11. A system, the system comprising:
a plurality of computing devices configured to distribute metadata into a plurality of buckets, wherein:
a bucket, of the plurality of buckets, is associated with a unique group of computing devices of the plurality of computing devices,
one computing device of the unique group of computing devices is selected as a leader of the bucket associated with the unique group, and
access to a particular address space is allowed only by the leader of the bucket associated with the unique group of computing devices.
12. The system of claim 11 , wherein the plurality of computing devices comprises a plurality of virtual file system (VFS) nodes.
13. The system of claim 11 , wherein each unique group of computing devices comprises five VFS nodes of a plurality of VFS nodes.
14. The system of claim 11 , wherein all VFS nodes in a unique group know and agree on the leader of the bucket.
15. The system of claim 11 , wherein each address space is associated with only one leader at any given time.
16. The system of claim 11 , wherein each of the computing devices is a leader of multiple buckets.
17. The system of claim 11 , wherein in the event of a failure of one of the plurality of computing devices, metadata that was on the failed computing device is redistributed.
18. The system of claim 11 , wherein in the event of a change in a number of computing devices in the plurality of computing devices, metadata is redistributed according to a load value associated with each computing device in the plurality of computing devices.
19. The system of claim 11 , wherein group leadership is changed in the event of a load imbalance.
20. The system of claim 11 , wherein one or more buckets of the plurality of buckets are split and redistributed in the event of a load imbalance.