System for queuing backup operations in a deduplicating storage system
A system for queuing deduplication backup operations comprises an input interface and a processor. The input interface configured to receive a request to perform a deduplication backup operation. The processor is configured to: a) determine a backup operation queue; b) determine whether the deduplication backup operation is at the head of the backup operation queue; c) determine whether the deduplication backup operation is qualified to be executed; and d) in the event the deduplication backup operation is qualified to be executed, execute the deduplication backup operation.
1. A system for queuing deduplication backup operations, comprising:
an input interface configured to receive a request to perform a deduplication backup operation;
a processor configured to:
determine a backup operation queue, wherein the backup operation queue comprises a plurality of types of backup operations, and wherein at least one of the plurality of types of backup operations comprises the deduplication backup operation;
determine whether the deduplication backup operation is at the head of the backup operation queue;
in response to determining that the deduplication backup operation is at the head of the backup operation queue, determine whether the deduplication backup operation is qualified to be executed based at least in part on a backup system load, wherein the backup system load is determined based at least in part on one or more thresholds, and at least one of the one or more thresholds is associated with an input stream load, and the one or more thresholds comprise a processor load threshold, the processor load threshold being determined based at least in part on a deduplication backup operation type; and
in response to determining that the deduplication backup operation is qualified to be executed, execute the deduplication backup operation.
2. The system of claim 1 , wherein the deduplication backup operation comprises a maintenance operation.
3. The system of claim 1 , wherein the deduplication backup operation comprises a storage operation.
4. The system of claim 1 , wherein the deduplication backup operation comprises a recover operation.
5. The system of claim 1 , wherein the processor is further configured to log the request to perform a deduplication backup operation.
6. The system of claim 1 , wherein the processor is further configured to load the deduplication backup operation into the backup operation queue.
7. The system of claim 1 , wherein determining whether the deduplication backup operation is qualified to be executed comprises checking one or more thresholds.
8. The system of claim 1 , wherein executing the deduplication backup operation comprises communicating with the requesting system.
9. The system of claim 8 , wherein communicating with the requesting system comprises requesting an input data stream.
10. The system of claim 8 , wherein communicating with the requesting system comprises throttling an input data stream.
11. The system of claim 8 , wherein communicating with the requesting system comprises providing a recovered file.
12. The system of claim 8 , wherein communicating with the requesting system comprises indicating a maintenance operation is complete.
13. The system of claim 1 , wherein the processor is further configured to log the execution of the deduplication backup operation.
14. The system of claim 1 , wherein determining whether the deduplication backup operation is qualified comprises determining whether an input stream number is below an input stream threshold.
15. The system of claim 1 , wherein determining whether the deduplication backup operation is qualified comprises determining whether a processor loading is below a processor threshold.
16. The system of claim 1 , wherein determining whether the deduplication backup operation is qualified comprises determining whether a storage activity is below a storage threshold.
17. A method for queuing deduplication backup operations, comprising:
receiving a request to perform a deduplication backup operation;
determining a backup operation queue, wherein the backup operation queue comprises a plurality of types of backup operations, and wherein at least one of the plurality of types of backup operations comprises the deduplication backup operation;
determining whether the deduplication backup operation is at the head of the backup operation queue;
in response to determining that the deduplication operation is at the head of the backup operation queue, determining whether the deduplication backup operation is qualified to be executed, based at least in part on a backup system load, wherein the backup system load is determined based at least in part on one or more thresholds, and at least one of the one or more thresholds is associated with an input stream load, and the one or more thresholds comprise a processor load threshold, the processor load threshold being determined based at least in part on a deduplication backup operation type; and
in response to determining that the deduplication backup operation is qualified to be executed, executing the deduplication backup operation.
18. A computer program product for queuing deduplication backup operations, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:
receiving a request to perform a deduplication backup operation;
determining a backup operation queue, wherein the backup operation queue comprises a plurality of types of backup operations, and wherein at least one of the plurality of types of backup operations comprises the deduplication backup operation;
determining whether the deduplication backup operation is at the head of the backup operation queue;
in response to determining that the deduplication backup operation is at the head of the backup operation queue, determining whether the deduplication backup operation is qualified to be executed based at least in part on a backup system load, wherein the backup system load is determined based at least in part on one or more thresholds, and at least one of the one or more thresholds is associated with an input stream load, and the one or more thresholds comprise a processor load threshold, the processor load threshold being determined based at least in part on a deduplication backup operation type; and
in response to determining that the deduplication backup operation is qualified to be executed, executing the deduplication backup operation.
19. The system of claim 1 , wherein the plurality of types of backup operations include backup storage operations, backup recovery operations, and backup maintenance operations.
20. The system of claim 19 , wherein the plurality of types of backup operations are queued together.
21. The system of claim 1 , wherein the backup system load is determined based at least in part on a current number of operations running.
22. The system of claim 1 , wherein at least one of the one or more thresholds is a processor load threshold.
23. The system of claim 1 , wherein a first deduplication backup operation type has a different processor load threshold than a second deduplication backup operation type.
24. The system of claim 21 , wherein the current number of operations running is determined based at least in part on processing log information to count started operations that have not been completed.
25. The system of claim 1 , wherein the processor is further configured to throttle a speed at which backup operations are obtained from the backup operation queue based on the backup system load.
26. The system of claim 1 , wherein the processor is further configured to determine a point in the backup operation queue to which a backup operation is to be added based at least in part on a corresponding deduplication backup operation priority.