Data protection component scaling in a cloud- based data storage system
Systems described herein may dynamically add one or more proxy data protection agents to a cloud data storage system to process a data protection job. Upon completion of the job or at some other appropriate interval, the system can power down and decommission the proxy data protection agents and/or the virtual machines on which the data protection proxies reside according to a cleanup schedule (e.g., at hourly or minute intervals). In order to improve the allocation of computing resources, the system takes into account currently existing proxies or virtual machines when processing a backup request to determine the need for new proxies to service the backup request. In this manner the system can save costs and computing resources through efficient virtual machine deployment and retirement.
1. An information management apparatus comprising:
one or more first computing devices in networked communication with a group of computing resources, the one or more first computing devices configured to:
based on job information associated with a first information management job request, calculate a first number of agents to install within the group of computing resources, the job information identifying a plurality of clients within the group of computing resources for which to perform information management operations; and
cause transmission of one or more commands to the group of computing resources, wherein the one or more commands include commands to:
install the first number of agents on one or more virtual machines within the group of computing resources; and
assign each agent of the first number of agents to a client of the plurality of clients to direct performance of information management operations associated with the client of the plurality of clients.
2. The information management apparatus of claim 1 , wherein the group of computing resources comprises a group of cloud resources, and wherein the one or more first computing devices cause the transmission of the one or more commands by interacting with an application programming interface.
3. The information management apparatus of claim 1 , wherein the one or more first computing devices comprise a resource scaling manager implemented on a first computing device of the one or more first computing devices and further comprises a storage manager implemented on a second computing device of the one or more first computing devices, and wherein the storage manager transmits the first information management job request to the resource scaling manager.
4. The information management apparatus of claim 1 , wherein a first agent of the first number of agents is installed on a first virtual machine of the one or more virtual machines, and wherein the one or more first computing devices are further configured to cause the first virtual machine to power off or shut down in response to one or both of an activity level of the first agent and an activity level of the first virtual machine.
5. The information management apparatus of claim 4 , wherein the one or more first computing devices are further configured to cause decommissioning of the first virtual machine.
6. The information management apparatus of claim 4 , wherein the one or more first computing devices are further configured to cause the first virtual machine to return from power off or shut down to an active state.
7. The information management apparatus of claim 1 , wherein, at a point in time the one or more first computing devices analyze the job information, a second number of agents is already instantiated within the group of computing resources, and wherein the one or more first computing devices use the second number of agents in calculating the first number of agents.
8. The information management apparatus of claim 1 , wherein the one or more first computing devices use a total number of clients of the plurality of clients in calculating the first number of agents.
9. The information management apparatus of claim 1 , wherein the one or more first computing devices use a data size associated with the first information management job request and an agent throughput value in calculating the first number of agents.
10. The information management apparatus of claim 1 wherein each of the first number of agents comprises at least one data agent and at least one media agent.
11. A method comprising:
with a management subsystem implemented in a computing device that is in networked communication with a group of computing resources:
analyzing job information associated with a first information management job request, the job information identifying at least a plurality of clients for which to perform information management operations, the plurality of clients implemented on one or more first computing devices of the group of computing resources;
based on the job information, calculating a first number of agents to install on the one or more first computing devices; and
causing transmission of one or more commands to the group of computing resources, wherein the one or more commands include commands to:
install the first number of agents on one or more virtual machines within the group of computing resources; and
assign each agent of the first number of agents to a client of the plurality of clients to direct performance of information management operations associated with the client of the plurality of clients.
12. The method of claim 11 , wherein the group of computing resources comprises a group of cloud resources, and wherein said causing the transmission of the one or more commands is achieved by interacting with an application programming interface.
13. The method of claim 11 , wherein a first agent of the first number of agents is installed on a first virtual machine of the one or more virtual machines, and the method further comprising causing the first virtual machine to power off or shut down in response to one or both of an activity level of the first agent and an activity level of the first virtual machine.
14. The method of claim 13 , further comprising causing decommissioning of the first virtual machine.
15. The method of claim 13 , further comprising causing the first virtual machine to return to an active state.
16. The method of claim 11 , wherein, at a point in time the management subsystem analyzes the job information, a second number of agents is already instantiated within the group of computing resources, and wherein said calculating comprises using the second number of agents to calculate the first number of agents.
17. The method of claim 11 , wherein said calculating comprises using a total number of clients of the plurality of clients to calculate the first number of agents.
18. The method of claim 11 , wherein said calculating comprises using a data size associated with the first information management job request and an agent throughput value to calculate the first number of agents.
19. An information management apparatus comprising:
one or more first computing devices in networked communication with a group of computing resources, the one or more first computing devices configured to:
based on job information associated with a first information management job request, calculate a first number of agents to install within the group of computing resources, the job information identifying a plurality of clients within the group of computing resources for which to perform information management operations;
install the first number of agents on one or more virtual machines within the group of computing resources; and
assign each agent of the first number of agents to a client of the plurality of clients to direct performance of information management operations associated with the client of the plurality of clients.