Method and system for workload aware storage replication
A method and system for workload aware storage replication. Specifically, the disclosed method and system entail establishing preferential transports for critical workload data targeted for replication from one cluster infrastructure to another. Existing replication solutions fail to prioritize the replication of critical over non-critical workload data, which the disclosed method and system rectify.
1. A method for workload data replication, comprising:
detecting a trigger event for a replication operation targeting a plurality of workloads hosted across a primary cluster infrastructure;
populating, in response to detecting the trigger event, a replication queue using a plurality of workload identifiers (IDs) for the plurality of workloads;
adjusting a replication order of the plurality of workload IDs in the replication queue based on a workload criticality class assigned to each workload;
replicating, based on the replication order, active workload data for the plurality of workloads to a secondary cluster infrastructure; and
prior to replicating the active workload data to the secondary cluster infrastructure:
adjusting the replication order based further on a number of incoming connections identified for each workload and an active workload data size associated with the active workload data for each workload.
2. The method of claim 1 , wherein the workload criticality class for each workload is derived from at least one selected from a group consisting of the number of incoming connections and the active workload data size.
3. The method of claim 1 , wherein the replication order is adjusted based further on the active workload data size when at least two workloads are tied reflecting a same number of incoming connections.
4. The method of claim 3 , wherein the at least two workloads share a same workload criticality class.
5. The method of claim 1 , wherein replicating the active workload data to the secondary cluster infrastructure, comprises:
allocating, based at least on the workload criticality class, a preferential transport size for each workload of the plurality of workloads; and
replicating, to the secondary cluster infrastructure, the active workload data for the workload using the preferential transport size.
6. The method of claim 5 , wherein the preferential transport size comprises network bandwidth.
7. A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to:
detect a trigger event for a replication operation targeting a plurality of workloads hosted across a primary cluster infrastructure;
populate, in response to detecting the trigger event, a replication queue using a plurality of workload identifiers (IDs) for the plurality of workloads;
adjust a replication order of the plurality of workload IDs in the replication queue based on a workload criticality class assigned to each workload;
replicate, based on the replication order, active workload data for the plurality of workloads to a secondary cluster infrastructure; and
prior to replicate the active workload data to the secondary cluster infrastructure:
adjust the replication order based further on a number of incoming connections identified for each workload and an active workload data size associated with the active workload data for each workload.
8. The non-transitory CRM of claim 7 , wherein the workload criticality class for each workload is derived from at least one selected from a group consisting of the number of incoming connections and the active workload data size.
9. The non-transitory CRM of claim 7 , wherein the replication order is adjusted based further on the active workload data size when at least two workloads are tied reflecting a same number of incoming connections.
10. The non-transitory CRM of claim 9 , wherein the at least two workloads share a same workload criticality class.
11. The non-transitory CRM of claim 7 , comprising computer readable program code, which when executed by the computer processor, further enables the computer processor to:
replicate the active workload data to the secondary cluster infrastructure, by:
allocating, based at least on the workload criticality class, a preferential transport size for each workload of the plurality of workloads; and
replicating, to the secondary cluster infrastructure, the active workload data for the workload using the preferential transport size.
12. The non-transitory CRM of claim 11 , wherein the preferential transport size comprises network bandwidth.
13. A system, comprising:
primary and secondary cluster infrastructures;
a computer processor interposed between and operatively connected to the primary and second cluster infrastructures; and
a workload data replicator executing on the computer processor, and programmed to:
detect a trigger event for a replication operation targeting a plurality of workloads hosted across the primary cluster infrastructure;
populate, in response to detecting the trigger event, a replication queue using a plurality of workload identifiers (IDs) for the plurality of workloads;
adjust a replication order of the plurality of workload IDs in the replication queue based on a workload criticality class assigned to each workload; and
replicate, based on the replication order, active workload data for the plurality of workloads to the secondary cluster infrastructure; and
prior to replicate the active workload data to the secondary cluster infrastructure:
adjust the replication order based further on a number of incoming connections identified for each workload and an active workload data size associated with the active workload data for each workload.
14. The system of claim 13 , further comprising:
a database operatively connected to the workload data replicator,
wherein a workload mapping for each workload is maintained on the database and comprises a workload ID for the workload, the workload criticality class assigned to the workload, and workload storage metadata of the active workload data associated with the workload.
15. The system of claim 14 , further comprising:
a criticality and dependency mapper executing on the computer processor, and operatively connected to the database,
wherein the storage metadata for each workload is identified by the criticality and dependency mapper based on input-output (IO) operations directed to the active workload data associated with the workload.
16. The system of claim 15 , wherein the primary cluster infrastructure, comprises:
a plurality of cluster nodes across which the plurality of workloads are hosted;
shared cluster storage comprising a plurality of storage devices across which the active workload data for the plurality of workloads are stored; and
a workload IO monitor interposed between the plurality of cluster nodes and the shared storage cluster, and configured to assist the criticality and dependency mapper in identifying the storage metadata for each workload.