Metadata control in a load-balanced distributed storage system
A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. The metadata is grouped into buckets. Each bucket is stored in a group of computing devices. However, only the leader of the group is able to directly access a particular bucket at any given time.
1. A method for controlling memory access, comprising:
distributing metadata into a plurality of buckets, wherein:
each bucket is associated with a unique group of computing devices of a plurality of computing devices;
the computing devices of each unique group dynamically select upon a particular computing device as a leader of the bucket associated with the unique group; and
allowing access to a particular address space only by a leader of a bucket to which metadata associated with that particular address space has been distributed.
2. The method of claim 1 , wherein the plurality of computing devices comprises a plurality of virtual file system (VFS) nodes.
3. The method of claim 1 , wherein each unique group of computing devices comprises five VFS nodes of a plurality of VFS nodes.
4. The method of claim 2 , wherein all VFS nodes in a unique group know and agree on a leader.
5. The method of claim 1 , wherein each address space has only one leader at any given time.
6. The method of claim 1 , wherein each of the computing devices is a leader of multiple buckets.
7. The method of claim 1 , wherein the method comprises, in the event of a failure of one of the plurality of computing devices, redistributing metadata that was on the failed computing device.
8. The method of claim 1 , wherein the method comprises, in the event of a change in a number of computing devices in the plurality of computing devices, redistributing metadata according to a load value associated with each computing device in the plurality of computing devices.
9. The method of claim 1 , wherein the method comprises changing group leadership in the event of a load imbalance.
10. The method of claim 1 , wherein the method comprises splitting and redistributing buckets of the plurality of buckets in the event of a load imbalance.
11. A system, the system comprising:
a plurality of computing devices; and
a plurality of buckets configured to store metadata, wherein:
each bucket is associated with a unique group of computing devices of the plurality of computing devices,
one computing device of each unique group of computing devices is selected as a leader of the bucket associated with the unique group, and
access to a particular address space is allowed only by a leader of a unique group of computing devices associated with a bucket that stores metadata associated with the particular address space.
12. The system of claim 11 , wherein the plurality of computing devices comprises a plurality of virtual file system (VFS) nodes.
13. The system of claim 11 , wherein each unique group of computing devices comprises five VFS nodes of a plurality of VFS nodes.
14. The system of claim 12 , wherein all VFS nodes in a unique group know and agree on a leader.
15. The system of claim 11 , wherein each address space is associated with only one leader at any given time.
16. The system of claim 11 , wherein each of the computing devices is a leader of multiple buckets.
17. The system of claim 11 , wherein in the event of a failure of one of the plurality of computing devices, metadata that was on the failed computing device is redistributed.
18. The system of claim 11 , wherein in the event of a change in a number of computing devices in the plurality of computing devices, metadata is redistributed according to a load value associated with each computing device in the plurality of computing devices.
19. The system of claim 11 , wherein group leadership is changed in the event of a load imbalance.
20. The system of claim 11 , wherein one or more buckets of the plurality of buckets are split and redistributed in the event of a load imbalance.