Dynamic management of effective bandwidth of data storage operations
Intelligent data throttling in data movement operations, such as secondary-copy operations in a storage management system. A local throttling manager may intelligently interoperate with co-resident data agents and/or a media agent executing on any given local computing device, whether a client computing device or a secondary storage computing device. The local throttling manager may allocate and manage the available bandwidth for various jobs and their constituent data streams—across the data agents and/or media agent. Effective bandwidth for the secondary-copy operation may be adjusted based on available bandwidth from the computing device due to increased demand for the bandwidth from other operations.
1. A non-transitory computer-readable storage medium storing instructions that when executed by one or more computing devices cause the one or more computing devices to perform a method for reducing effective bandwidth for a data storage operation executing on a source computing device, the method comprising:
executing the data storage operation on the source computing device using at least some of available bandwidth of the source computing device to transmit data to a destination computing device in a storage system,
wherein the data storage operation is executed according to a schedule indicated in a storage policy assigned to the source computing device,
wherein the storage policy comprises a collection of settings or preferences for performing backup operations on data stored on the source computing device;
determining an effective bandwidth utilized by the data storage operation at the source computing device;
reducing the effective bandwidth for the data storage operation on the source computing device according to settings or preferences specified in the storage policy assigned to the source computing device;
receiving data from the source computing device via one or more data streams; and
storing the received data in the storage system.
2. The non-transitory computer-readable storage medium of claim 1 further comprising monitoring one or more performance metrics of the source computing device.
3. The non-transitory computer-readable storage medium of claim 2 , wherein reducing the effective bandwidth for the data storage operation is based at least in part on the one or more performance metrics satisfying a threshold value associated with the one or more performance metrics.
4. The non-transitory computer-readable storage medium of claim 3 , wherein the threshold value associated with the one or more performance metrics is specified by the storage policy.
5. The non-transitory computer-readable storage medium of claim 2 , wherein the one or more performance metrics includes a latency metric associated with the source computing device.
6. The non-transitory computer-readable storage medium of claim 2 , wherein the one or more performance metrics includes a latency metric associated with the source computing device.
7. The non-transitory computer-readable storage medium of claim 1 , wherein the reducing the effective bandwidth for the data storage operation is executed by a throttling manager.
8. The non-transitory computer-readable storage medium of claim 1 , wherein reducing the effective bandwidth for the data storage operation includes re-allocating data streams to or from the source computing device.
9. The non-transitory computer-readable storage medium of claim 1 , wherein the method further comprises performing deduplication operations on the received data according to the storage policy.
10. The non-transitory computer-readable storage medium of claim 1 , wherein reducing the effective bandwidth for the data storage operation includes dynamically reducing the effective bandwidth without pausing or restarting the data storage operation.
11. The non-transitory computer-readable storage medium of claim 1 , wherein the source computing device executes operations associated with providing services hosted by the source computing device.
12. The non-transitory computer-readable storage medium of claim 1 , wherein the destination computing device is hosted by a cloud service provider.
13. The non-transitory computer-readable storage medium of claim 1 , wherein the data storage operation is a snapshot operation, a replication operation, an archive operation, or a data migration operation.
14. An information management system configured to reduce effective bandwidth for a data storage operation executing on a source computing device, the information management system comprising:
one or more computing devices comprising computer hardware configured to:
execute the data storage operation on the source computing device using at least some of available bandwidth of the source computing device to transmit data to a destination computing device in a storage system,
wherein the data storage operation is executed according to a schedule indicated in a storage policy assigned to the source computing device,
wherein the storage policy comprises a collection of settings or preferences for performing backup operations on data stored on the source computing device;
determine an effective bandwidth utilized by the data storage operation at the source computing device;
reduce the effective bandwidth for the data storage operation on the source computing device according to settings or preferences specified in the storage policy assigned to the source computing device;
receive data from the source computing device via one or more data streams; and
store the received data in the storage system.
15. The information management system of claim 14 , wherein the one or more computing devices is further configured to monitor one or more performance metrics of the source computing device.
16. The information management system of claim 15 , wherein reducing the effective bandwidth for the data storage operation is based at least in part on the one or more performance metrics satisfying a threshold value associated with the one or more performance metrics.
17. The information management system of claim 16 , wherein the threshold value associated with the one or more performance metrics is specified by the storage policy.
18. The information management system of claim 15 , wherein the one or more performance metrics includes a latency metric associated with the source computing device.
19. The information management system of claim 15 , wherein the one or more performance metrics includes a latency metric associated with the source computing device.
20. The information management system of claim 14 , wherein the one or more computing devices is further configured to re-allocate data streams to or from the source computing device.
21. The information management system of claim 14 , wherein the reducing the effective bandwidth for the data storage operation is executed by a throttling manager.
22. The information management system of claim 14 , wherein the one or more computing devices is further configured to perform deduplication operations on the received data according to the storage policy.
23. The information management system of claim 14 , wherein reducing the effective bandwidth for the data storage operation includes dynamically reducing the effective bandwidth without pausing or restarting the data storage operation.
24. The information management system of claim 14 , wherein the source computing device executes operations associated with providing services hosted by the source computing device.
25. The information management system of claim 14 , wherein the destination computing device is hosted by a cloud service provider.
26. The information management system of claim 14 , wherein the data storage operation is a snapshot operation, a replication operation, an archive operation, or a data migration operation.