Replicating data with a data orchestrator

1 . A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations for:

receiving a replication request and a policy that indicates a number of times an original dataset is to be replicated on a plurality of storage systems as a plurality of corresponding replicated datasets, wherein at least one storage system of the plurality of storage systems is a custom storage system type and at least another storage system of the plurality of storage systems is a non-custom storage system type; and

for each storage system of the plurality of storage systems and a corresponding replicated dataset of the plurality of corresponding replicated datasets,

in response to determining that the storage system is the custom storage system type,

executing a custom storage system plug-in associated with the storage system to generate a custom script by translating the policy into a custom set of instructions; and

executing the custom script to create the corresponding replicated dataset on the storage system; and

in response to determining that the storage system is the non-custom storage system type,

generating a non-custom script by translating the policy into a non-custom set of instructions; and

executing the non-custom script to create the corresponding replicated dataset on the storage system.

2 . The computer program product of claim 1 , wherein the program instructions are executable by the processor to cause the processor to perform further operations for:

storing the policy and metadata for the original dataset in a local data store.

3 . The computer program product of claim 1 , wherein the program instructions are executable by the processor to cause the processor to perform further operations for:

receiving an updated policy that indicates another number of times the original dataset is to be replicated;

determining that the number of times the original dataset is to be replicated in the policy does not match another number of times the original dataset is to be replicated in the updated policy; and

automatically updating the plurality of corresponding replicated datasets based on the updated policy by performing at least one of creating a new replicated dataset and deleting an existing replicated dataset by generating at least one new script by translating the updated policy into a new set of instructions.

4 . The computer program product of claim 1 , wherein the program instructions are executable by the processor to cause the processor to perform further operations for:

identifying the plurality of storage systems based on the number of times the original dataset is to be replicated, available capacity at the plurality of storage systems, and data locality of the plurality of storage systems.

5 . The computer program product of claim 1 , wherein the program instructions are executable by the processor to cause the processor to perform further operations for:

periodically scanning the plurality of corresponding replicated datasets to check for compliance with the policy;

determining that a replicated dataset of the plurality of corresponding replicated datasets is on a storage system of the plurality of storage systems that is no longer available; and

creating a new replicated dataset on another storage system of the plurality of storage systems that is available.

6 . The computer program product of claim 1 , wherein the policy is created using a declarative policy language and comprises the number of times the original dataset is to be replicated, replication locations, replication modes, replication frequency, system affinities, security rules, and governance rules.

7 . The computer program product of claim 1 , wherein the storage systems comprise any combination of edge nodes, cloud nodes, on-prem nodes, and file system nodes.

8 . A computer system, comprising:

one or more processors, one or more computer-readable memories and one or more computer-readable, tangible storage devices; and

program instructions, stored on at least one of the one or more computer-readable, tangible storage devices for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, to perform operations comprising:

receiving a replication request and a policy that indicates a number of times an original dataset is to be replicated on a plurality of storage systems as a plurality of corresponding replicated datasets, wherein at least one storage system of the plurality of storage systems is a custom storage system type and at least another storage system of the plurality of storage systems is a non-custom storage system type; and

for each storage system of the plurality of storage systems and a corresponding replicated dataset of the plurality of corresponding replicated datasets,

in response to determining that the storage system is the custom storage system type,

executing a custom storage system plug-in associated with the storage system to generate a custom script by translating the policy into a custom set of instructions; and

executing the custom script to create the corresponding replicated dataset on the storage system; and

in response to determining that the storage system is the non-custom storage system type,

generating a non-custom script by translating the policy into a non-custom set of instructions; and

executing the non-custom script to create the corresponding replicated dataset on the storage system.

9 . The computer system of claim 8 , wherein the program instructions perform further operations comprising:

storing the policy and metadata for the original dataset in a local data store.

10 . The computer system of claim 8 , wherein the program instructions perform further operations comprising:

receiving an updated policy that indicates another number of times the original dataset is to be replicated;

determining that the number of times the original dataset is to be replicated in the policy does not match another number of times the original dataset is to be replicated in the updated policy; and

automatically updating the plurality of corresponding replicated datasets based on the updated policy by performing at least one of creating a new replicated dataset and deleting an existing replicated dataset by generating at least one new script by translating the updated policy into a new set of instructions.

11 . The computer system of claim 8 , wherein the program instructions perform further operations comprising:

identifying the plurality of storage systems based on the number of times the original dataset is to be replicated, available capacity at the plurality of storage systems, and data locality of the plurality of storage systems.

12 . The computer system of claim 8 , wherein the program instructions perform further operations comprising:

periodically scanning the plurality of corresponding replicated datasets to check for compliance with the policy;

determining that a replicated dataset of the plurality of corresponding replicated datasets is on a storage system of the plurality of storage systems that is no longer available; and

creating a new replicated dataset on another storage system of the plurality of storage systems that is available.

13 . The computer system of claim 8 , wherein the policy is created using a declarative policy language and comprises the number of times the original dataset is to be replicated, replication locations, replication modes, replication frequency, system affinities, security rules, and governance rules.

14 . The computer system of claim 8 , wherein the storage systems comprise any combination of edge nodes, cloud nodes, on-prem nodes, and file system nodes.

15 . A computer-implemented method, comprising operations for:

receiving a replication request and a policy that indicates a number of times an original dataset is to be replicated on a plurality of storage systems as a plurality of corresponding replicated datasets, wherein at least one storage system of the plurality of storage systems is a custom storage system type and at least another storage system of the plurality of storage systems is a non-custom storage system type; and

for each storage system of the plurality of storage systems and a corresponding replicated dataset of the plurality of corresponding replicated datasets,

in response to determining that the storage system is the custom storage system type,

executing a custom storage system plug-in associated with the storage system to generate a custom script by translating the policy into a custom set of instructions; and

executing the custom script to create the corresponding replicated dataset on the storage system; and

in response to determining that the storage system is the non-custom storage system type,

generating a non-custom script by translating the policy into a non-custom set of instructions; and

executing the non-custom script to create the corresponding replicated dataset on the storage system.

16 . The computer-implemented method of claim 15 , further comprising operations for:

storing the policy and metadata for the original dataset in a local data store.

17 . The computer-implemented method of claim 15 , further comprising operations for:

receiving an updated policy that indicates another number of times the original dataset is to be replicated;

determining that the number of times the original dataset is to be replicated in the policy does not match another number of times the original dataset is to be replicated in the updated policy; and

automatically updating the plurality of corresponding replicated datasets based on the updated policy by performing at least one of creating a new replicated dataset or deleting an existing replicated dataset by generating at least one new script by translating the updated policy into a new set of instructions.

18 . The computer-implemented method of claim 15 , further comprising operations for:

identifying the plurality of storage systems based on the number of times the original dataset is to be replicated, available capacity at the plurality of storage systems, and data locality of the plurality of storage systems.

19 . The computer-implemented method of claim 15 , further comprising operations for:

periodically scanning the plurality of corresponding replicated datasets to check for compliance with the policy;

determining that a replicated dataset of the plurality of corresponding replicated datasets is on a storage system of the plurality of storage systems that is no longer available; and

creating a new replicated dataset on another storage system of the plurality of storage systems that is available.

20 . The computer-implemented method of claim 15 , wherein the policy is created using a declarative policy language and comprises the number of times the original dataset is to be replicated, replication locations, replication modes, replication frequency, system affinities, security rules, and governance rules.