Provision and configuration of quorum witnesses in clusters
A cloud managed storage system ( 200 ) includes a cluster of the nodes ( 220 ) that are already in communication with a cloud management plane ( 280 ) that has visibility of all the storage nodes ( 220 ) in the cluster. When a cluster is created, the number of nodes ( 220 ) is determined and based on the number of nodes ( 220 ), the storage system ( 200 ) can determine whether an external quorum witness ( 284 ) is required and automatically enable a quorum witness ( 284 ) as a cloud service. Storage nodes ( 220 ) can contact the quorum witness ( 284 ) when necessary for failover processes.
1 . A process comprising:
identifying, by a management system, a first storage node and a second storage node of a cluster storage system;
automatically creating, by the management system, a quorum witness service as a cloud service in a management plane of the cluster storage system based on a number of storage nodes in the cluster storage system, wherein the quorum witness service has visibility to available storage nodes which are outside the management plane; and
configuring each of the first storage node and the second storage node to contact the quorum witness service when the first or the second storage node cannot contact a quorum of the available storage nodes in the cluster storage system.
2 . The process of claim 1 , further comprising:
configuring, by the management system, the first storage node and the second storage node to provide storage services of the cluster storage system.
3 . The process of claim 2 , further comprising:
configuring, by the management system, the first storage node to present one or more first virtual volumes for storage services from the cluster storage system; and
configuring, by the management system, the second storage node to maintain one or more first backup volumes that respectively copy the one or more first virtual volumes, wherein the first storage node and the second storage node communicate to maintain the first backup volumes as copies of the first virtual volumes.
4 . The process of claim 3 , further comprising:
configuring, by the management system, the second storage node to present one or more second virtual volumes for storage services from the cluster storage system; and
configuring, by the management system, the first storage node to maintain one or more second backup volumes that respectively copy the one or more second virtual volumes, wherein the first storage node and the second storage node communicate to maintain the second backup volumes as copies of the second virtual volumes.
5 . The process of claim 3 , further comprising:
attempting, by the second storage node, to contact the quorum witness service in response to the second storage node detecting a failure of communication with the first storage node, wherein the contact with the quorum witness service is attempted using a network system.
6 . The process of claim 5 , further comprising:
entering, by the second storage node, a failed state in response to the second storage node determining that the contact attempted has failed.
7 . The process of claim 5 , further comprising:
providing, by the quorum witness service, node visibility information to the second storage node in response to receiving a request from the second storage node;
indicating, using the node visibility information, which of the available storage nodes the quorum witness service can access; and
in response to the node visibility information indicating that the quorum witness service cannot access the first storage node, using, by the second storage node, the first backup volumes to present the first virtual volumes for the storage services that the storage cluster system provides.
8 . The process of claim 7 , further comprising:
in response to the node visibility information indicating that the quorum witness service can access the first storage node, determining, by the second storage node, whether the first storage node or the second storage node has priority;
in response to determining that the first storage node has priority, entering, by the second storage node, a failed state; and
in response to determining that the second storage node has priority, using, by the second storage node, the first backup volumes to present the first virtual volumes for the storage services from the storage cluster system.
9 . The process of claim 1 , wherein configuring each of the first and second storage nodes to contact the quorum witness service further comprises:
enabling the first storage node to communicate with the quorum witness service through a network system; and
enabling the second storage node to communicate with the quorum witness service through a same or a different network system.
10 . The process of claim 9 , further comprising:
configuring, by the management system, the first storage node and the second storage node to provide storage services of the cluster storage system.
11 . A storage system comprising a processing unit configured to control backend storage to provide storage services in a cluster storage system, the processing unit comprising:
a first communication interface adapted for first communications with further processing units configured to provide storage services of one or more other nodes of the cluster storage system; and
a second communication interface adapted for second communications through a network, the second communications with a quorum witness service in a management plane and which has visibility to available storage nodes which are outside the management plane, the quorum witness service automatically created based on a number of the available storage nodes in the cluster storage system, the second communications in response to a failure of the first communications with the one or more other storage nodes through the first communication interface.
12 . The storage system of claim 11 , wherein the processing unit is further configured to transmit a request to a management system in the management plane and wherein the management system, in response to the request, configures a first one of the available storage nodes and a second one of the available storage nodes to provide storage services of the cluster storage system.
13 . The storage system of claim 12 , wherein the processing unit is further configured to transmit a request to a management system in the management plane and wherein the management system configures the first one of the available storage nodes to present one or more first virtual volumes for storage services from the cluster storage system, and configures for the second one of the available storage nodes to maintain one or more first backup volumes that respectively copy the one or more first virtual volumes, wherein the first one of the available storage nodes and the second one of the available storage nodes communicate to maintain the first backup volumes as copies of the first virtual volumes.
14 . The storage system of claim 13 , wherein the processing unit is further configured to transmit a request to a management system in the management plane and wherein the management system configures the first one of the available storage nodes to present one or more second virtual volumes for storage services from the cluster storage system, and configures the second one of the available storage nodes to maintain one or more second backup volumes that respectively copy the one or more second virtual volumes, wherein the first one of the available storage nodes and the second one of the available storage nodes communicate to maintain the one or more second backup volumes as copies of the second virtual volumes.
15 . The storage system of claim 13 , the second one of the available storage nodes is to attempt to contact the quorum witness service in response to the second one of the available storage nodes detecting a failure of communication with the first one of the available storage nodes, wherein the contact with the quorum witness service is to be attempted using a network system.
16 . The storage system of claim 15 , wherein the second one of the available storage nodes is to enter a failed state in response to the second one of the available storage nodes determining that the contact attempted has failed.
17 . The storage system of claim 15 , wherein:
the quorum witness service is to provide node visibility information to the second one of the available storage nodes in response to receiving the request from the second one of the available storage nodes;
the node visibility information is to indicate which of the available storage nodes the quorum witness service can access; and
in response to the node visibility information indicating that the quorum witness service cannot access the first one of the available storage nodes, the second one of the available storage nodes to use the first backup volumes to present the first virtual volumes for the storage services from the storage cluster system.
18 . The storage system of claim 17 , wherein:
in response to the node visibility information indicating that the quorum witness service can access the first one of the available storage nodes, the second one of the available storage nodes is to determine whether the first one of the available storage nodes or the second one of the available storage nodes has priority;
in response to determining that the first one of the storage nodes has priority, the second one of the available storage nodes is to enter a failed state; and
in response to determining that the second one of the available storage nodes has priority, the second one of the available storage nodes is to use the first backup volumes to present the first virtual volumes for the storage services from the storage cluster system.
19 . The storage system of claim 11 , wherein a first one of the available storage nodes is to communicate with the quorum witness service through a network system, and wherein a second one of the available storage nodes is to communicate with the quorum witness service through the same or a different network system.
20 . The storage system of claim 19 , wherein a management system of the management plane is to configure the first one of the available storage nodes and the second one of the available storage nodes to provide storage services of the cluster storage system.