Prioritizing I/O operations directed to replicating datasets
Providing Quality of Service (QoS) for replicating datasets including: receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository and the target data repository; adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues for the target data repository; selecting, by the target data repository, one or more queues from the plurality of queues; and servicing an operation from each of the selected one or more queues.
1 . A method for providing Quality of Service (QoS) for replicating datasets comprising:
receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository;
adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues that includes one or more other queues that have a higher priority than the first queue, the one or more other queues configured to queue storage operations from a host that are directed to at least one volume; and
servicing, by the target data repository, an operation from one or more queues of the plurality of queues in preference to a priority of servicing the first queue.
2 . The method of claim 1 , wherein the plurality of queues includes one or more second queues each corresponding to a particular volume of the target data repository and each configured to queue storage operations directed to the particular volume.
3 . The method of claim 2 , further comprising:
receiving, by the target data repository, a storage operation directed to volume of the target data repository;
adding, by the target data repository, the storage operation to a second queue of the one or more second queues corresponding to the volume.
4 . The method of claim 1 , wherein further comprising selecting the one or more queues based on a respective accumulated cost of serviced operations for each of the plurality of queues.
5 . The method of claim 4 , further comprising updating, for each of the one or more queues, the respective accumulated cost.
6 . The method of claim 5 , wherein the respective accumulated cost is updated based on a priority value based on an entity type.
7 . The method of claim 6 , wherein the priority value is further based on a user-defined weight.
8 . The method of claim 1 , further comprising:
sending, by the target data repository to the source data repository, data describing a load of the target data repository; and
wherein the data causes the source data repository to adjust one or more parameters for modifying the one or more datasets in the source data repository.
9 . The method of claim 1 , further comprising indicating, by the target data repository, one or more limits on receiving checkpoints from the source data repository, wherein the one or more limits cause the source data repository to adjust at least one operation associated with checkpoints at the source data repository.
10 . A storage system for providing Quality of Service (QoS) for replicating datasets that includes a computer memory and a computer processor, the computer memory including program instructions that, when executed by the computer processor, cause the storage system to carry out the following steps of:
receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository;
adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues that includes one or more other queues that have a higher priority than the first queue, the one or more other queues configured to queue storage operations from a host that are directed to at least one volume; and
servicing, by the target data repository, an operation from one or more queues of the plurality of queues in preference to a priority of servicing the first queue.
11 . The storage system of claim 10 , wherein the plurality of queues includes one or more second queues each corresponding to a particular volume of the target data repository and each configured to queue storage operations directed to the particular volume.
12 . The storage system of claim 11 , wherein the program instructions, when executed by the computer processor, further cause the storage system to carry out the following steps of:
receiving, by the target data repository, a storage operation directed to volume of the target data repository;
adding, by the target data repository, the storage operation to a second queue of the one or more second queues corresponding to the volume.
13 . The storage system of claim 10 , wherein selecting the program instructions, when executed by the computer processor, further cause the storage system to select the one or more queues based on a respective accumulated cost of serviced operations for each of the plurality of queues.
14 . The storage system of claim 13 , wherein the program instructions, when executed by the computer processor, further cause the storage system to carry out the following step of updating, for each of the one or more queues, the respective accumulated cost.
15 . The storage system of claim 14 , wherein the respective accumulated cost is updated based on a priority value based on an entity type.
16 . The storage system of claim 15 , wherein the priority value is further based on a user-defined weight.
17 . The storage system of claim 10 , wherein the program instructions, when executed by the computer processor, further cause the storage system to carry out the following steps of:
sending, by the target data repository to the source data repository, data describing a load of the target data repository; and
wherein the data causes the source data repository to adjust one or more parameters for modifying the one or more datasets in the source data repository.
18 . The storage system of claim 10 , wherein the program instructions, when executed by the computer processor, further cause the storage system to carry out the following step of modifying, by the source data repository, a data replication mode in response to an age of a last serviced checkpoint exceeding a threshold.
19 . A computer program product disposed on a non-transitory computer readable medium, the computer program product including computer program instructions for providing Quality of Service (QoS) for replicating datasets that, when executed, carry out the following steps of:
receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository;
adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues that includes one or more other queues that have a higher priority than the first queue, the one or more other queues configured to queue storage operations from a host that are directed to at least one volume; and
servicing, by the target data repository, an operation from one or more queues of the plurality of queues in preference to a priority of servicing the first queue.
20 . The computer program product of claim 19 , wherein the plurality of queues includes one or more second queues each corresponding to a particular volume of the target data repository and each configured to queue storage operations directed to the particular volume.