Hierarchical input/output fencing in clustered environments
View Patent ↗Disclosed herein are methods, systems, and processes to perform hierarchical fencing in clustered environments. A first set of registration keys is received from a first set of nodes that are supported by a first hypervisor and are part of a cluster. A first key value is generated based on the first set of registration keys. The first key value is then stored on a shared storage device shared by the first set of nodes and a second set of nodes supported by a second hypervisor. Upon receiving a request to perform an input/output (I/O) fencing operation, the first key value is modified and the second key value is removed from the shared storage device.
1. A method comprising:
receiving a request to perform an input/output (I/O) fencing operation, wherein
a first key value, generated from a first set of registration keys received from a first set of nodes of a cluster, is stored in a shared storage device,
the first set of nodes are supported by a first hypervisor,
the shared storage device is shared by the first set of nodes and a second set of nodes, and
the second set of nodes are supported by a second hypervisor; and
in response to receipt of the request,
determining whether a node of the first set of nodes is subject to the I/O fencing operation, and
in response to a determination that the node is subject to the I/O fencing operation,
modifying the first key value by removing a key value associated with the node, and
performing a persistent reservation operation on the node, wherein
the persistent reservation operation comprises at least one of an unregister operation, a reserve operation, or a preempt operation.
2. The method of claim 1 , further comprising:
generating the first key value based, at least in part, on the first set of registration keys; and
storing the first key value on the shared storage device.
3. The method of claim 1 , further comprising:
determining that a second node has joined the cluster, wherein
the second set of nodes comprise the second node;
determining that the second node joining the cluster has caused
storing of a second key value in the shared storage device, and
removal of the first key value from the shared storage device;
determining whether the second node is subject to the I/O fencing operation; and
in response to a determination that the second node is subject to the I/O fencing operation,
performing one or more persistent reservation operations on the second node.
4. The method of claim 3 , wherein
the one or more persistent reservation operations comprise at least one of a register operation, an unregister operation, a reserve operation, or a preempt operation.
5. The method of claim 1 , further comprising:
determining whether a second node is subject to the I/O fencing operation, wherein the second set of nodes comprise the second node; and
in response to a determination that the second node is subject to the I/O fencing operation, removing a second key value from the shared storage device, wherein the second key value is associated with the second set of nodes.
6. The method of claim 5 , wherein
the shared storage device is a Small Computer System Interface 3 (SCSI-3) storage device, and
the first hypervisor and the second hypervisor access a Logical Unit Number (LUN) of the SCSI-3 disk as part of performing the persistent reservation operation.
7. The method of claim 1 , wherein
the first hypervisor and the second hypervisor implement a first hierarchical fencing engine and a second hierarchical fencing engine, respectively,
the first key value corresponds to one or more nodes executing on the first hypervisor,
a second key value corresponds to one or more nodes executing on the second hypervisor and is stored by the second hypervisor on the shared storage device,
the removing the first key value indicates to the first hypervisor that the node is subject to the I/O fencing operation,
the first hypervisor is configured to cause the second hypervisor to update the second key value, as a result of the first hypervisor updating the first key value, and
the second hypervisor is configured to cause the first hypervisor to update the first key value, as a result of the first hypervisor updating the second key value.
8. A non-transitory computer readable storage medium comprising program instructions that, when executed by one or more processors of a computing system, are configured to cause at least one of the one or more processors to:
receive a request to perform an input/output (I/O) fencing operation, wherein
a first key value, generated from a first set of registration keys received from a first set of nodes of a cluster, is stored in a shared storage device,
the first set of nodes are supported by a first hypervisor,
the shared storage device is shared by the first set of nodes and a second set of nodes, and
the second set of nodes are supported by a second hypervisor; and
in response to receipt of the request,
determine whether a node of the first set of nodes is subject to the I/O fencing operation, and
in response to a determination that the node is subject to the I/O fencing operation,
modify the first key value by removing a key value associated with the node, and
perform a persistent reservation operation on the node, wherein
the persistent reservation operation comprises at least one of an unregister operation, a reserve operation, or a preempt operation.
9. The non-transitory computer readable storage medium of claim 8 , wherein the program instructions further comprise program instructions that, when executed by the one or more processors, are configured to cause at least one of the one or more processors to:
generate the first key value based, at least in part, on the first set of registration keys; and
store the first key value on the shared storage device.
10. The non-transitory computer readable storage medium of claim 8 , wherein the program instructions further comprise program instructions that, when executed by the one or more processors, are configured to cause at least one of the one or more processors to:
determine that a second node has joined the cluster, wherein
the second set of nodes comprise the second node;
determine that the second node joining the cluster has caused
storing of a second key value in the shared storage device, and
removal of the first key value from the shared storage device;
determine whether the second node is subject to the I/O fencing operation; and
in response to a determination that the second node is subject to the I/O fencing operation,
perform one or more persistent reservation operations on the second node.
11. The non-transitory computer readable storage medium of claim 10 , wherein
the one or more persistent reservation operations comprise at least one of a register operation, an unregister operation, a reserve operation, or a preempt operation.
12. The non-transitory computer readable storage medium of claim 8 , wherein the program instructions further comprise program instructions that, when executed by the one or more processors, are configured to cause at least one of the one or more processors to:
determine whether a second node is subject to the I/O fencing operation, wherein the second set of nodes comprise the second node; and
in response to a determination that the second node is subject to the I/O fencing operation, remove a second key value from the shared storage device, wherein the second key value is associated with the second set of nodes.
13. The non-transitory computer readable storage medium of claim 12 , wherein
the shared storage device is a Small Computer System Interface 3 (SCSI-3) storage device, and
the first hypervisor and the second hypervisor access a Logical Unit Number (LUN) of the SCSI-3 disk as part of performing the persistent reservation operation.
14. The non-transitory computer readable storage medium of claim 8 , wherein
the first hypervisor and the second hypervisor implement a first hierarchical fencing engine and a second hierarchical fencing engine, respectively,
the first key value corresponds to one or more nodes executing on the first hypervisor,
a second key value corresponds to one or more nodes executing on the second hypervisor and is stored by the second hypervisor on the shared storage device,
removal of the first key value indicates to the first hypervisor that the node is subject to the I/O fencing operation,
the first hypervisor is configured to cause the second hypervisor to update the second key value, as a result of the first hypervisor updating the first key value, and
the second hypervisor is configured to cause the first hypervisor to update the first key value, as a result of the first hypervisor updating the second key value.
15. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions that, when executed by the one or more processors, are configured to cause at least one of the one or more processors to
receive a request to perform an input/output (I/O) fencing operation, wherein
a first key value, generated from a first set of registration keys received from a first set of nodes of a cluster, is stored in a shared storage device,
the first set of nodes are supported by a first hypervisor,
the shared storage device is shared by the first set of nodes and a second set of nodes, and
the second set of nodes are supported by a second hypervisor,
in response to receipt of the request,
determine whether a node of the first set of nodes is subject to the I/O fencing operation, and
in response to a determination that the node is subject to the I/O fencing operation,
modify the first key value by removing a key value associated with the node, and
perform a persistent reservation operation on the node, wherein
the persistent reservation operation comprises at least one of an unregister operation, a reserve operation, or a preempt operation.
16. The system of claim 15 , wherein the program instructions further comprise program instructions executable that, when executed by the one or more processors, are configured to cause the at least one of the one or more processors to:
generate the first key value based, at least in part, on the first set of registration keys; and
store the first key value on the shared storage device.
17. The system of claim 15 , wherein the program instructions further comprise program instructions that, when executed by the one or more processors, are configured to cause the at least one of the one or more processors to:
determine that a second node has joined the cluster, wherein
the second set of nodes comprise the second node;
determine that the second node joining the cluster has caused
storing of a second key value in the shared storage device, and
removal of the first key value from the shared storage device;
determine whether the second node is subject to the I/O fencing operation; and
in response to a determination that the second node is subject to the I/O fencing operation,
perform one or more persistent reservation operations on the second node.
18. The system of claim 17 , wherein
the one or more persistent reservation operations comprise at least one of a register operation, an unregister operation, a reserve operation, or a preempt operation.
19. The system of claim 15 , wherein the program instructions further comprise program instructions executable that, when executed by the one or more processors, are configured to cause at least one of the one or more processors to:
determine whether a second node is subject to the I/O fencing operation, wherein the second set of nodes comprise the second node; and
in response to a determination that the second node is subject to the I/O fencing operation, remove a second key value from the shared storage device, wherein the second key value is associated with the second set of nodes.
20. The system of claim 19 , wherein
the shared storage device is a Small Computer System Interface 3 (SCSI-3) storage device, and
the first hypervisor and the second hypervisor access a Logical Unit Number (LUN) of the SCSI-3 disk as part of performing the persistent reservation operation.
21. The system of claim 15 , wherein
the first hypervisor and the second hypervisor implement a first hierarchical fencing engine and a second hierarchical fencing engine, respectively,
the first key value corresponds to one or more nodes executing on the first hypervisor,
a second key value corresponds to one or more nodes executing on the second hypervisor and is stored by the second hypervisor on the shared storage device,
removal of the first key value indicates to the first hypervisor that the node is subject to the I/O fencing operation,
the first hypervisor is configured to cause the second hypervisor to update the second key value, as a result of the first hypervisor updating the first key value, and
the second hypervisor is configured to cause the first hypervisor to update the first key value, as a result of the first hypervisor updating the second key value.