Active membership in a cloud-based storage system
Determining active membership among a set of storage systems, including: determining, by a cloud-based storage system among the set of storage systems, that a membership event corresponds to a change in membership to the set of storage systems synchronously replicating the dataset; applying, in dependence upon the membership event, one or more membership protocols to determine a new set of storage systems to synchronously replicate the dataset; and for one or more I/O operations directed to the dataset, applying the one or more I/O operations to the dataset synchronously replicated by the new set of storage systems.
1. A method comprising:
determining, by a cloud-based storage system among a set of storage systems synchronously replicating a dataset, that a membership event corresponds to a change in membership to the set of storage systems synchronously replicating the dataset;
applying, in dependence upon the membership event, one or more membership protocols to determine a new set of storage systems to synchronously replicate the dataset; and
for one or more I/O operations directed to the dataset, applying the one or more I/O operations to the dataset synchronously replicated by the new set of storage systems, wherein storage systems synchronously replicating a dataset are included in a membership list of in-sync members that is respectively stored on each of the set of storage systems, and wherein in-sync members correspond to a management entity representing the dataset, a set of managed objects, and a set of management operations.
2. The method of claim 1 , wherein the cloud-based storage system includes a plurality of cloud computing instances that respectively store a portion of the dataset.
3. The method of claim 1 , wherein the plurality of cloud computing instances is included in a list of cloud computing instance members.
4. The method of claim 3 , wherein responsive to a membership event among the plurality of cloud computing instances, one or more of the cloud computing instances apply the one or more membership protocols to determine a new set of cloud computing instances to store a complete version of the dataset.
5. The method of claim 1 , wherein the one or more membership protocols include a quorum protocol, an external management protocol, or a racing protocol.
6. The method of claim 5 , wherein each storage system in the list of in-sync members corresponds to a respective quantity of votes, and wherein the quorum protocol specifies that a majority of votes from a subset of the storage systems is required for the subset of storage systems to detach a storage system that is not a member of the subset of storage systems.
7. The method of claim 5 , wherein the external management protocol specifies that, in response to determining the membership event, each individual storage system of the set of storage systems request permission from a member storage system of the in-sync list of members prior to attaching or detaching a given storage system from the in-sync list of members, wherein the member storage system comprises a highly available cluster, a distributed configuration tool, or a distributed database.
8. The method of claim 5 , wherein the racing protocol specifies that, in response to determining the membership event, each individual storage system of the set of storage systems attempt to access an exclusively accessible resource provided by a computer system that is not included in the in-sync list of members, and wherein a storage system that gains access to the exclusively accessible resource is able to, to the exclusion of other storage systems in the in-sync list of members, propagate a membership change for the in-sync list of members.
9. The method of claim 5 , wherein the racing protocol is a mediation protocol for mediating between storage systems.
10. A cloud-based storage system comprising:
one or more cloud computing instances with local block storage;
cloud-based object storage; and
one or more storage controller applications, each storage controller application executing in a cloud computing instance, wherein the one or more storage controllers are configured to:
determine that a membership event corresponds to a change in membership to a set of storage systems synchronously replicating the dataset;
apply, in dependence upon the membership event, one or more membership protocols to determine a new set of storage systems to synchronously replicate the dataset; and
for one or more I/O operations directed to the dataset, apply the one or more I/O operations to the dataset synchronously replicated by the new set of storage systems, wherein storage systems synchronously replicating a dataset are included in a membership list of in-sync members that is respectively stored on each of the set of storage systems, and wherein in-sync members correspond to a management entity representing the dataset, a set of managed objects, and a set of management operations.
11. The cloud-based storage system of claim 10 , wherein the one or more cloud computing instances respectively store a portion of the dataset.
12. The cloud-based storage system of claim 10 , wherein the one or more cloud computing instances are included in a list of cloud computing instance members.
13. The cloud-based storage system of claim 12 , wherein responsive to a membership event among the one or more cloud computing instances, one or more of the cloud computing instances apply the one or more membership protocols to determine a new set of cloud computing instances to store a complete version of the dataset.
14. The cloud-based storage system of claim 10 , wherein the one or more membership protocols include a quorum protocol, an external management protocol, or a racing protocol.
15. The cloud-based storage system of claim 14 , wherein each storage system in the list of in-sync members corresponds to a respective quantity of votes, and wherein the quorum protocol specifies that a majority of votes from a subset of the storage systems is required for the subset of storage systems to detach a storage system that is not a member of the subset of storage systems.
16. The cloud-based storage system of claim 14 , wherein the external management protocol specifies that, in response to determining the membership event, each individual storage system of the set of storage systems request permission from a member storage system of the in-sync list of members prior to attaching or detaching a given storage system from the in-sync list of members, wherein the member storage system comprises a highly available cluster, a distributed configuration tool, or a distributed database.
17. The cloud-based storage system of claim 14 , wherein the racing protocol specifies that, in response to determining the membership event, each individual storage system of the set of storage systems attempt to access an exclusively accessible resource provided by a computer system that is not included in the in-sync list of members, and wherein a storage system that gains access to the exclusively accessible resource is able to, to the exclusion of other storage systems in the in-sync list of members, propagate a membership change for the in-sync list of members.
18. A method comprising:
determining, by a hardware-based storage system in communication with a cloud-based storage system, that a membership event at the cloud-based storage system corresponds to a change in membership to a set of storage systems synchronously replicating a dataset;
applying, in dependence upon the membership event, one or more membership protocols to determine a new set of storage systems to synchronously replicate the dataset; and
for one or more I/O operations directed to the dataset, applying the one or more I/O operations to the dataset synchronously replicated by the new set of storage systems, wherein storage systems synchronously replicating a dataset are included in a membership list of in-sync members that is respectively stored on each of the set of storage systems, and wherein in-sync members correspond to a management entity representing the dataset, a set of managed objects, and a set of management operations.