Partition arbitration optimization
Various systems, methods, and processes to optimize partition arbitration are disclosed. In response to a cluster being partitioned into multiple network partitions, a determination is made, using a first node in a first network partition, whether the first node wins one or more coordination points. The determination of the first node is then transmitted to a second node in the first network partition. Further, another determination is made using the second node whether the second node wins one or more other coordination points other than the one or more coordination points won by the first node.
1. A computer-implemented method comprising:
in response to a cluster being partitioned into a plurality of network partitions,
determining, using a first node in a first network partition of the plurality of network partitions, whether
the first node wins a race between the first node and another node, for one or more coordination points of a plurality of coordination points, wherein the first node, in response to winning the race, can configure the one or more coordination points to fence off the other node in the cluster in order to preclude access by the other node to one or more storage devices, and
the plurality of coordination points comprise an odd number of coordination points,
transmitting the determination of the first node to a second node in the first network partition, and
determining, using the second node, whether the second node wins one or more additional coordination points of the plurality of coordination points, other than the one or more coordination points.
2. The computer-implemented method of claim 1 , wherein
the determining using the second node further comprises
the first node and the second node together winning a majority of coordination points of the plurality of coordination points.
3. The computer-implemented method of claim 1 , wherein
winning the one or more coordination points comprises
accessing the one or more coordination points;
ejecting a registration key of the other node, which is not in the first network partition from the one or more coordination points;
wherein ejecting the registration key effectively preempts the other node from winning the one or more coordination points.
4. The computer-implemented method of claim 2 , further comprising
transmitting the determination of the second node to a third node, if the first node and the second node together do not win the majority of coordination points of the plurality of coordination points, wherein
the third node is part of the first network partition.
5. The computer-implemented method of claim 1 , wherein
the determining using the first node comprises performing, using the first node, a first fencing race, and
the determining using the second node comprises performing, using the second node, a second fencing race.
6. The computer-implemented method of claim 1 , wherein
the determining using the first node or the determining using the second node comprises accessing a matrix, wherein
the matrix is a coordination point reachability matrix (CPRM), and
the CPRM is replicated across all nodes of a plurality of nodes in the cluster.
7. The computer-implemented method of claim 1 , wherein
the plurality of coordination points comprise
one or more coordinator disks, or
one or more coordination point servers.
8. The computer-implemented method of claim 1 , wherein
the first node is designated as a first racer node, and
after the first racer node transmits the determination to the second node,
the second node is designated as a second racer node.
9. A non-transitory computer readable storage medium storing program instructions executable to:
in response to a cluster being partitioned into a plurality of network partitions,
determine, using a first node in a first network partition of the plurality of network partitions, whether
the first node wins a race between the first node and another node, for one or more coordination points of a plurality of coordination points, wherein the first node, in response to winning the race, can configure the one or more coordination points to fence off the other node in the cluster in order to preclude access by the other node to one or more storage devices, and
the plurality of coordination points comprise an odd number of coordination points,
transmit the determination of the first node to a second node in the first network partition, and
determine, using the second node, whether the second node wins one or more additional coordination points of the plurality of coordination points, other than the one or more coordination points.
10. The non-transitory computer readable storage medium of claim 9 , wherein
the determining using the second node further comprises
the first node and the second node together winning a majority of coordination points of the plurality of coordination points.
11. The non-transitory computer readable storage medium of claim 9 , wherein
winning the one or more coordination points comprises
accessing the one or more coordination points;
ejecting a registration key of the other node, which is not in the first network partition from the one or more coordination points;
wherein ejecting the registration key effectively preempts the other node from winning the one or more coordination points.
12. The non-transitory computer readable storage medium of claim 10 , further comprising
transmitting the determination of the second node to a third node, if the first node and the second node together do not win the majority of coordination points of the plurality of coordination points, wherein
the third node is part of the first network partition.
13. The non-transitory computer readable storage medium of claim 9 , wherein
the determining using the first node comprises performing, using the first node, a first fencing race, and
the determining using the second node comprises performing, using the second node, a second fencing race.
14. The non-transitory computer readable storage medium of claim 9 , wherein
the determining using the first node or the determining using the second node comprises accessing a matrix, wherein
the matrix is a coordination point reachability matrix (CPRM), and
the CPRM is replicated across all nodes of a plurality of nodes in the cluster.
15. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to:
in response to a cluster being partitioned into a plurality of network partitions,
determine, using a first node in a first network partition of the plurality of network partitions, whether
the first node wins a race between the first node and another node, for one or more coordination points of a plurality of coordination points, wherein the first node, in response to winning the race, can configure the one or more coordination points to fence off the other node in the cluster in order to preclude access by the other node to one or more storage devices, and
the plurality of coordination points comprise an odd number of coordination points,
transmit the determination of the first node to a second node in the first network partition, and
determine, using the second node, whether the second node wins one or more additional coordination points of the plurality of coordination points, other than the one or more coordination points.
16. The system of claim 15 , wherein
the determining using the second node further comprises
the first node and the second node together winning a majority of coordination points of the plurality of coordination points.
17. The system of claim 15 , wherein
winning the one or more coordination points comprises
accessing the one or more coordination points;
ejecting a registration key of the other node, which is not in the first network partition from the one or more coordination points;
wherein ejecting the registration key effectively preempts the other node from winning the one or more coordination points.
18. The system of claim 16 , further comprising
transmitting the determination of the second node to a third node, if the first node and the second node together do not win the majority of coordination points of the plurality of coordination points, wherein
the third node is part of the first network partition.
19. The system of claim 15 , wherein
the determining using the first node comprises performing, using the first node, a first fencing race, and
the determining using the second node comprises performing, using the second node, a second fencing race.
20. The system of claim 15 , wherein
the first node is designated as a first racer node, and
after the first racer node transmits the determination to the second node,
the second node is designated as a second racer node.