Storage tier selection for replication and recovery
Disclosed herein are methods, systems, and processes to select storage tiers for data replication and data recovery. A replication stream is received from a replication appliance that is configured to replicate data to a storage tier of multiple storage tiers. Each storage tier differs from at least one other storage tier in at least one storage characteristic. Portions of the replication stream are identified based on input/output (I/O) characteristics. The portions are stored in one storage tier other than the storage tier based on the I/O characteristics, and a storage cost associated with each storage tier, where the storage cost associated with each storage tier is based on storage characteristics of each storage tier.
1. A computer-implemented method comprising:
receiving a replication stream and storage tier selection information at a target replication appliance, wherein
the replication stream and the storage tier selection information are received from a source replication appliance,
the replication stream comprises a plurality of update sets,
the replication stream represents only write operations received at the source replication appliance,
each update set of the plurality of update sets represents one or more write operations of the write operations received at the source replication appliance,
the storage tier selection information identifies a plurality of storage tiers in which each of the plurality of update sets is to be stored, and
each storage tier of the plurality of storage tiers is of a corresponding one of a plurality of storage types;
identifying a storage type of a first storage tier of the plurality of storage tiers, using at least a portion of the storage tier selection information, wherein
at least one update set of the plurality of update sets is to be stored in a storage unit in the first storage tier, and
the storage type is one of the plurality of storage types;
determining whether a consolidation operation can be performed on the at least one update set, based, at least in part, on the storage type;
in response to a determination that the consolidation operation can be performed,
producing a consolidated update set by performing a consolidation operation on the at least one update set, wherein
the consolidation operation consolidates a first write operation of a first update set of the plurality of update sets and a second write operation of a second update set of the plurality of update sets into the consolidated update set, and
storing the consolidated update set in a storage unit of the first storage tier; and
in response to a determination that the consolidation operation cannot be performed,
storing at least one update set of the plurality of update sets in the storage unit.
2. The computer-implemented method of claim 1 , wherein
the plurality of storage tiers comprises at least one of
a plurality of storage types,
a plurality of storage systems, or
a plurality of storage devices.
3. The computer-implemented method of claim 1 , wherein
the plurality of storage tiers are associated with the source replication appliance.
4. The computer-implemented method of claim 1 , further comprising:
receiving reconfiguration instructions at the target replication appliance to facilitate storage of replicated data to at least one of the plurality of storage tiers other than the first storage tier.
5. The computer-implemented method of claim 1 , further comprising:
storing a first portion of the replication stream to a second storage tier of the plurality of storage tiers, wherein
the second storage tier is other than the first storage tier, and
storing a second portion of the replication stream to a third storage tier of the plurality of storage tiers, wherein
the second storage tier is other than the first storage tier.
6. The computer-implemented method of claim 1 , wherein
one or more characteristics are associated with the replication stream, and
the one or more characteristics comprise, at least in part, one or more of
an I/O rate,
a size of an I/O, or
a locality of reference.
7. The computer-implemented method of claim 1 , wherein
a storage tier of the plurality of storage tiers comprises one or more block storage devices, and
another storage tier of the plurality of storage tiers comprises one or more object storage devices.
8. The computer-implemented method of claim 7 wherein
a service level agreement is associated with the replication stream,
the service level agreement comprises a plurality of constraints, and
the plurality of constraints comprise at least in part, a recovery point objective and a recovery time objective associated with the replication stream.
9. The computer-implemented method of claim 8 , further comprising:
determining whether the recovery point objective and the recovery time objective indicate that the at least one update set should be stored in another storage tier of the plurality of storage tiers, wherein
the another storage tier is other than the first storage tier,
the determining is performed without regard to a storage cost associated with the another storage tier being higher than the storage cost associated with the first storage tier.
10. The computer-implemented method of claim 1 , further comprising:
determining whether a virtual machine of a plurality of virtual machines can be supported by the each storage tier, wherein
the determining is based, at least in part, on at least one of a number of disks that can be coupled to a cloud gateway, a maximum data size on a local area network, or an I/O rate of each virtual machine of the plurality of virtual machines.
11. The computer-implemented method of claim 10 , further comprising:
calculating a storage cost of replicating another replication stream received from the virtual machine based on an average I/O rate of the plurality of virtual machines.
12. A non-transitory computer-readable storage medium, comprising program instructions, which, when executed by one or more processors of a computing system, perform a method comprising:
receiving a replication stream and storage tier selection information at a target replication appliance, wherein
the replication stream and the storage tier selection information are received from a source replication appliance,
the replication stream comprises a plurality of update sets,
the replication stream represents only write operations received at the source replication appliance,
each update set of the plurality of update sets represents one or more write operations of the write operations received at the source replication appliance,
the storage tier selection information identifies a plurality of storage tiers in which each of the plurality of update sets is to be stored, and
each storage tier of the plurality of storage tiers is of a corresponding one of a plurality of storage types;
identifying a storage type of a first storage tier of the plurality of storage tiers, using at least a portion of the storage tier selection information, wherein
at least one update set of the plurality of update sets is to be stored in a storage unit in the first storage tier, and
the storage type is one of the plurality of storage types;
determining whether a consolidation operation can be performed on the at least one update set, based, at least in part, on the storage type;
in response to a determination that the consolidation operation can be performed,
producing a consolidated update set by performing a consolidation operation on the at least one update set, wherein
the consolidation operation consolidates a first write operation of a first update set of the plurality of update sets and a second write operation of a second update set of the plurality of update sets, and
storing the consolidated update set in a storage unit of the first storage tier; and
in response to a determination that the consolidation operation cannot be performed,
storing at least one update set of the plurality of update sets in the storage unit.
13. The non-transitory computer-readable storage medium of claim 12 , wherein
the plurality of storage tiers comprises at least one of
a plurality of storage types,
a plurality of storage systems, or
a plurality of storage devices.
14. The non-transitory computer-readable storage medium of claim 12 , wherein the method further comprises:
storing a first portion of the replication stream to a second storage tier of the plurality of storage tiers, wherein
the second storage tier is other than the first storage tier, and
storing a second portion of the replication stream to a third storage tier of the plurality of storage tiers, wherein
the second storage tier is other than the first storage tier.
15. The non-transitory computer-readable storage medium of claim 12 , wherein
a storage tier of the plurality of storage tiers comprises one or more block storage devices, and
another storage tier of the plurality of storage tiers comprises one or more object storage devices.
16. The non-transitory computer-readable storage medium of claim 15 , wherein
a service level agreement is associated with the replication stream,
the service level agreement comprises a plurality of constraints, and
the plurality of constraints comprise at least in part, a recovery point objective and a recovery time objective associated with the replication stream.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the method further comprises:
determining whether the recovery point objective and the recovery time objective indicate that the at least one update set should be stored in another storage tier of the plurality of storage tiers, wherein
the another storage tier is other than the first storage tier,
the determining is performed without regard to a storage cost associated with the another storage tier being higher than the storage cost associated with the first storage tier.
18. The non-transitory computer-readable storage medium of claim 12 , wherein the method further comprises:
determining whether a virtual machine of a plurality of virtual machines can be supported by the each storage tier, wherein
the determining is based, at least in part, on at least one of a number of disks that can be coupled to a cloud gateway, a maximum data size on a local area network, or an I/O rate of each virtual machine of the plurality of virtual machines.
19. The non-transitory computer-readable storage medium of claim 18 , wherein the method further comprises:
calculating a storage cost of replicating another replication stream received from the virtual machine based on an average I/O rate of the plurality of virtual machines.
20. A computing system comprising:
one or more processors; and
a computer-readable storage medium coupled to the one or more processors, comprising program instructions, which, when executed by the one or more processors, perform a method comprising
receiving a replication stream and storage tier selection information at a target replication appliance, wherein
the replication stream and the storage tier selection information are received from a source replication appliance,
the replication stream comprises a plurality of update sets,
the replication stream represents only write operations received at the source replication appliance,
each update set of the plurality of update sets represents one or more write operations of the write operations received at the source replication appliance,
the storage tier selection information identifies a plurality of storage tiers in which each of the plurality of update sets is to be stored, and
each storage tier of the plurality of storage tiers is of a corresponding one of a plurality of storage types,
identifying a storage type of a first storage tier of the plurality of storage tiers, using at least a portion of the storage tier selection information, wherein
at least one update set of the plurality of update sets is to be stored in a storage unit in the first storage tier, and
the storage type is one of the plurality of storage types;
determining whether a consolidation operation can be performed on the at least one update set, based, at least in part, on the storage type;
in response to a determination that the consolidation operation can be performed,
producing a consolidated update set by performing a consolidation operation on the at least one update set, wherein
the consolidation operation consolidates a first write operation of a first update set of the plurality of update sets and a second write operation of a second update set of the plurality of update sets, and
storing the consolidated update set in a storage unit of the first storage tier; and
in response to a determination that the consolidation operation cannot be performed, storing at least one update set of the plurality of update sets in the storage unit.