I/O scheduling and load balancing across the multiple nodes of a clustered environment utilizing data volume based scheduling priorities
Write operations are scheduled for multiple nodes in a shared storage cluster that supports volume replication. Requests are received from nodes for allocation of space for write operations in a replication log. In response to a received request, the current capacity of the requesting node to manage a backlog can be determined. The amount of space in the replication log allocated to the node is then calibrated to the node's capacity, thereby preventing self-throttling. A separate priority can be assigned to each volume, and space in the replication log assigned to each volume based on its priority. Nodes can target synchronous and other latency sensitive operations to higher priority volumes. A single global queue can be maintained to schedule write operations for all nodes, thereby providing a fair scheduling. A separate local queue can be maintained for each node, thereby providing specific levels of preference to specific nodes.
1. A computer implemented method for scheduling write operations of a plurality of nodes, thereby balancing a load of storage input/output operations in a clustering and storage environment that utilizes shared storage comprising a plurality of data volumes, wherein the clustering and storage environment supports replication of the data volumes, the method comprising the steps of:
receiving, by a computer, requests from nodes of the clustering and storage environment for allocation of space for write operations in a replication log, wherein the nodes perform write operations by writing first to the replication log and subsequently to the shared storage, wherein an order of the write operations to the replication log matches an order of write operations to the shared storage, and wherein the clustering and storage environment utilizes the order and content of the write operations to the replication log to replicate the data volumes of the shared storage to a secondary site, thereby preserving write-order fidelity;
assigning, by the computer, a scheduling priority to each one of the plurality of data volumes of the shared storage; and
allocating, by the computer, space in the replication log to the each one of the plurality of data volumes based on its assigned scheduling priority, such that write operations to data volumes with higher assigned scheduling priorities are processed at a higher level of priority than write operations to data volumes with lower assigned scheduling priorities.
2. The method of claim 1 further comprising:
targeting specific ones of the plurality of nodes targeting synchronous and other higher latency sensitive write operations to higher priority volumes, and targeting asynchronous and other lower priority write operations to lower priority volumes.
3. The method of claim 1 wherein the receiving the requests from the nodes of the clustering and storage environment for allocation of space for the write operations further comprises:
receiving the requests from the nodes of the clustering and storage environment for allocation of space for the write operations, each one of the received requests further comprises information indicative of a priority of a corresponding write operation; and
scheduling the write operations at a level of priority indicated in corresponding received requests.
4. The method of claim 1 further comprising:
receiving input entered by a user indicating a scheduling priority to assign a specific data volume; and
assigning the scheduling priority to the specific data volume in response to the received input.
5. The method of claim 1 further comprising:
receiving input entered by a user indicating an adjustment to make to a scheduling priority assigned to a specific data volume; and
adjusting the scheduling priority assigned to the specific data volume as indicated by the received input.
6. A non-transitory computer readable-storage medium for scheduling write operations of a plurality of nodes, thereby balancing a load of storage input/output operations in a clustering and storage environment that utilizes shared storage comprising a plurality of data volumes, wherein the clustering and storage environment supports replication of the data volumes, the non-transitory computer readable-storage medium storing computer executable instructions that, when loaded into computer memory and executed by at least one processor of a computing device, cause the computing device to perform the following steps:
receiving requests from nodes of the clustering and storage environment for allocation of space for write operations in a replication log, wherein the nodes perform write operations by writing first to the replication log and subsequently to the shared storage, wherein an order of the write operations to the replication log matches an order of write operations to the shared storage, and wherein the clustering and storage environment utilizes the order and content of the write operations to the replication log to replicate the data volumes of the shared storage to a secondary site, thereby preserving write-order fidelity;
assigning a scheduling priority to each one of the plurality of data volumes of the shared storage; and
allocating space in the replication log to the each one of the plurality of data volumes based on its assigned scheduling priority, such that write operations to data volumes with higher assigned scheduling priorities are processed at a higher level of priority than write operations to data volumes with lower assigned scheduling priorities.
7. The non-transitory computer readable-storage medium of claim 6 further comprising:
targeting specific ones of the plurality of nodes targeting synchronous and other higher latency sensitive write operations to higher priority volumes, and targeting asynchronous and other lower priority write operations to lower priority volumes.
8. The non-transitory computer readable-storage medium of claim 6 wherein the receiving the requests from the nodes of the clustering and storage environment for allocation of space for the write operations further comprises:
receiving the requests from the nodes of the clustering and storage environment for allocation of space for the write operations, each one of the received requests further comprises information indicative of a priority of a corresponding write operation; and
scheduling the write operations at a level of priority indicated in corresponding received requests.
9. The non-transitory computer readable-storage medium of claim 6 further comprising:
receiving input entered by a user indicating a scheduling priority to assign a specific data volume; and
assigning the scheduling priority to the specific data volume in response to the received input.
10. The non-transitory computer readable-storage medium of claim 6 further comprising:
receiving input entered by a user indicating an adjustment to make to a scheduling priority assigned to a specific data volume; and
adjusting the scheduling priority assigned to the specific data volume as indicated by the received input.