Metadata control in a load-balanced distributed storage system
A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. The metadata is grouped into buckets. Each bucket is stored in a group of computing devices. However, only the leader of the group is able to directly access a particular bucket at any given time.
1 . A method, comprising:
associating a reserved amount of memory with a unique group comprising a plurality of computing devices, prior to assignment of metadata for any particular address space;
selecting a particular computing device, of the unique group, as a leader of the unique group; and
allowing access to a particular address space only by the leader of the unique group by distributing metadata associated with the particular address space to the reserved amount of memory associated with the unique group such that non-leader computing devices of the unique group are prevented from directly accessing the particular address space.
2 . The method of claim 1 , wherein the unique group comprises a plurality of virtual file system (VFS) nodes.
3 . The method of claim 1 , wherein the unique group comprises five VFS nodes of a plurality of VFS nodes.
4 . The method of claim 1 , wherein:
the unique group comprises a plurality of VFS nodes, all VFS nodes of the unique group know and agree on the leader of the unique group.
5 . The method of claim 1 , wherein each address space has only one leader at any given time.
6 . The method of claim 1 , wherein the particular computing device is a leader of unique groups.
7 . The method of claim 1 , wherein the method comprises, in the event of a failure of one of a plurality of computing devices of a distributed storage system, redistributing metadata that was on the failed computing device.
8 . The method of claim 1 , wherein the method comprises, in the event of a change in a number of computing devices in the distributed storage system, redistributing metadata according to a load value associated with each computing device.
9 . The method of claim 1 , wherein the method comprises changing group leadership in the event of a load imbalance.
10 . The method of claim 1 , wherein the method comprises splitting and redistributing the memory in the event of a load imbalance.
11 . A system, the system comprising:
a reserved amount of memory which is pre-allocated independently of address-space assignment and is associated with a unique group comprising a plurality of computing devices,
one computing device of the unique group is selected as a leader of the unique group, and
access to a particular address space is allowed only by the leader of the unique group by storing metadata associated with the particular address space in the reserved amount of memory associated with the unique group such that other computing devices of the unique group are prevented from directly accessing the particular address space.
12 . The system of claim 11 , wherein the plurality of computing devices comprises a plurality of virtual file system (VFS) nodes.
13 . The system of claim 11 , wherein the unique group of computing devices comprises five VFS nodes of a plurality of VFS nodes.
14 . The system of claim 11 , wherein all VFS nodes in the unique group know and agree on the leader.
15 . The system of claim 11 , wherein each address space is associated with only one leader at any given time.
16 . The system of claim 11 , wherein each of the computing devices is a leader of multiple unique groups.
17 . The system of claim 11 , wherein in the event of a failure of one of the plurality of computing devices, metadata that was on the failed computing device is redistributed.
18 . The system of claim 11 , wherein in the event of a change in a number of computing devices in the plurality of computing devices, metadata is redistributed according to a load value associated with each computing device in the plurality of computing devices.
19 . The system of claim 11 , wherein group leadership is changed in the event of a load imbalance.
20 . The system of claim 11 , wherein the memory is split and redistributed in the event of a load imbalance.