Automation of data storage activities
A system receives data storage workflow activities that include computer-executable instructions for carrying out data storage workflow in a network data storage system. Once the workflow is received, the system deploys the workflow to one or more workflow engines that can execute the various data storage activities related to the workflow. Prior to executing a data storage activity, the system can determine which workflow engine to use based on an allocation scheme.
1. A method for allocating execution of a data management workflow suite within a networked data storage system, the method comprising:
receiving relationship data, wherein the relationship data indicates an order in which data management activities are to be performed for at least first and second client computing devices having one or more processors;
generating a data management workflow suite based at least in part on the received relationship data, the data management workflow suite comprising executable instructions for carrying out a plurality of data management activities;
deploying the data management workflow suite to a workflow engine which executes on a second computing system comprising one or more processors;
receiving an instruction to initiate the data management workflow suite;
based at least in part on the instruction to initiate the data management workflow suite, causing the workflow engine to execute one or more data management activities, wherein executing the one or more data management activities comprises:
identifying a first data agent for implementing at least one of the one or more data management activities to be performed for the first client computing device,
identifying a second data agent for implementing at least one of the one or more data management activities to be performed for the second client computing device, and
executing one or more data management activities to be performed for the first or second client computing devices; and
based at least in part on the relationship data of the data management workflow suite, identifying a second data management activity of the data management workflow suite to execute.
2. The method of claim 1 , wherein the plurality of data management activities further comprises one or more restore management activities in which data residing in a secondary storage is restored to a primary storage.
3. The method of claim 1 , the method further comprising selecting the workflow engine according to an allocation scheme.
4. The method of claim 3 , wherein the workflow engine is a first workflow engine, the method further comprising:
deploying the data management workflow suite to a second workflow engine which executes on a separate computing system comprising one or more processors.
5. The method of claim 4 , further comprising obtaining information related to activity levels associated with the workflow engine.
6. The method of claim 4 , wherein deploying the data management workflow suite to the first workflow engine and the second workflow engine is based at least in part on one or more of computing resources associated with the first workflow engine and one or more of computing resources associated with the second workflow engine and geographic proximity of one or more components in a networked data management system to the first workflow engine and the second workflow engine.
7. The method of claim 3 , wherein the allocation scheme and scheduling of the one or more data management activities are based, at least in part, on information related to activity levels associated with the first workflow engine and the second workflow engine.
8. The method of claim 5 , wherein the allocation scheme is based at least in part on at least one of:
computing resources associated with the first workflow engine and one or more of computing resources associated with the second workflow engine,
physical proximity of the first workflow engine and the second workflow engine to other data storage system components, or
communication pathway speeds between the other data storage system components and the first workflow engine and the second workflow engine.
9. The method of claim 7 , wherein the information related to activity levels associated with the first workflow engine and the second workflow engine comprises information related to at least one of:
a total number of workflow suites deployed on the first workflow engine and the second workflow engine,
a total number of processes active on the first workflow engine and the second workflow engine, and
utilization of the first workflow engine and the second workflow engine.
10. A data management system, comprising:
a first computing system executing on one or more processors and configured to:
receive relationship data, wherein the relationship data indicates an order in which data management activities are to be performed for at least first and second client computing devices having one or more processors;
generate a data management workflow suite based at least in part on the received relationship data, the data management workflow suite comprising executable instructions for carrying out a plurality of data management activities;
deploy the data management workflow suite to a workflow engine which executes on a second computing system comprising one or more processors;
receive an instruction to initiate the data management workflow suite;
based at least in part on the instruction to initiate the data management workflow suite, cause the workflow engine to execute one or more data management activities, wherein executing the one or more data management activities comprises:
identifying a first data agent for implementing at least one of the one or more data management activities to be performed for the first client computing device,
identifying a second data agent for implementing at least one of the one or more data management activities to be performed for the second client computing device, and
executing one or more data management activities to be performed for the first or second client computing devices; and
based at least in part on the relationship data of the data management workflow suite, identify a second data management activity of the data management workflow suite to execute.
11. The data management system of claim 10 , wherein the plurality of data management activities further comprises one or more restore management activities in which data residing in a secondary storage is restored to a primary storage.
12. The data management system of claim 10 , wherein the workflow engine is a first workflow engine, the first computing system further configured to:
deploy the data management workflow suite to a second workflow engine which executes on a separate computing system comprising one or more processors.
13. The data management system of claim 12 , further comprising obtaining information related to activity levels associated with the workflow engine.
14. The data management system of claim 13 , the first computing system further configured to: select the workflow engine according to an allocation scheme.
15. The data management system of claim 14 , wherein the allocation scheme and scheduling of the one or more data management activities are based, at least in part, on information related to activity levels associated with the first workflow engine and the second workflow engine.
16. The data management system of claim 14 , wherein deploying the data management workflow suite to the first workflow engine and the second workflow engine is based at least in part on one or more of computing resources associated with the first workflow engine and one or more of computing resources associated with the second workflow engine and geographic proximity of one or more components in the data management system to the first workflow engine and the second workflow engine.
17. The data management system of claim 14 , wherein the allocation scheme is based at least in part on at least one of:
computing resources associated with the first workflow engine and one or more of computing resources associated with the second workflow engine,
physical proximity of the first workflow engine and the second workflow engine to other data storage system components, or
communication pathway speeds between the other data storage system components and the first workflow engine and the second workflow engine.
18. The data management system of claim 15 , wherein the information related to activity levels associated with the first workflow engine and the second workflow engine comprises information related to at least one of:
a total number of workflow suites deployed on the first workflow engine and the second workflow engine,
a total number of processes active on the first workflow engine and the second workflow engine, and
utilization of the first workflow engine and the second workflow engine.