Predictive scheduled backup system and method
Embodiments for predictive scheduling of backups in a data protection system by initiating a first backup job in a series of scheduled consecutive backup jobs, wherein a second backup job is allowed to begin only after the first backup job is finished and not active, detecting whether or not the first backup job is still active when a second job is to start, and if so, estimating an amount of additional time required to finish the first backup job. The second backup job is then rescheduled to start at least at the end of the additional time. The estimated amount of additional time is determined using a throughput to target storage device parameter. This parameter is periodically checked to determine if there is a change to the estimated amount of additional time, and if so, the estimated time is recalculated based on the changed parameter.
1. A computer-implemented method of predictive scheduled backups of data in a data protection system using virtual machine (VM) based storage, comprising:
initiating a first backup job in a series of scheduled consecutive backup jobs, wherein a next backup job is allowed to begin only after the first backup job is finished and not active;
detecting, by the next backup job, if the first backup job is still active;
gathering, if the first backup job is still active, statistics of the first backup job to evaluate a time required to finish the first backup job including an amount of additional time to finish the first backup job; and
rescheduling the next backup job to start at least at the end of the additional time.
2. The method of claim 1 wherein the one or more statistics are gathered from a media database of a backup server functionally coupled to the virtual machines.
3. The method of claim 1 wherein the estimated amount of additional time is determined using a throughput to storage device parameter.
4. The method of claim 3 wherein the estimated amount of additional time is further determined through additional parameters including: type of the target device, network transfer rate, processor usage during the first backup, and an amount of data to be written for the first backup.
5. The method of claim 3 further comprising:
periodically checking the throughput to target device parameter for change from a defined threshold in excess of a defined threshold value; and
re-executing the estimating step to recalculate the estimated completion time if the change exceeds the defined threshold value.
6. The method of claim 3 wherein the data protection system comprises a data backup system implemented by a Networker backup server and the VM-based storage is implemented using a VMware-based hypervisor.
7. The method of claim 6 wherein the media database is a database maintained by NetWorker server to keep the detailed information such as which backup resides on which media and detailed statistics of the first and next backup jobs.
8. The method of claim 1 wherein the first backup is a full backup and the next backup is one of a differential or incremental backup.
9. The method of claim 8 wherein the full backup is followed by a plurality of incremental backups including the next backup according to a defined periodic schedule.
10. A computer-implemented method of predictive scheduled backups of data in a data protection system using virtual machine (VM) based storage, comprising:
initiating a backup job comprising a first backup followed by a plurality of subsequent backups according to a defined schedule in which a subsequent backup job does not start until an immediately preceding backup job is completed;
gathering statistics of the first backup job when the first backup job is active;
estimating a completion time of the first backup job based on the statistics;
determining if there is an overlap of the estimated completion time of the first backup job and a start time of a second backup job subsequent to the first backup job; and
if so, rescheduling the second backup job to start after the estimated completion time of the first backup job.
11. The method of claim 10 further comprising:
periodically checking for throughput changes;
performing a dynamic recalculation to modify the estimated completion time using a throughput parameter; and
re-estimating the completion time based on any change to the throughput parameter.
12. The method of claim 11 wherein the change to the throughput parameter comprises a deviation in excess of a defined threshold value.
13. The method of claim 12 the throughput parameter comprises a throughput to target device parameter.
14. The method of claim 13 wherein the statistics further comprise at least one of: type of the target device, network transfer rate, processor usage during the first backup, and an amount of data to be written for the first backup.
15. The method of claim 10 wherein the first backup is a full backup and the second backup is one of a differential or incremental backup.
16. A system for predictively scheduling backups of data in a data protection system using virtual machine (VM) based storage, comprising:
a backup server initiating a backup job comprising a first backup followed by a plurality of subsequent backups according to a defined schedule in which a subsequent backup job does not start until an immediately preceding backup job is completed;
a media database storing gathered statistics of the first backup job when the first backup job is active; and
a predictive scheduled backup component estimating a completion time of the first backup job based on the statistics, determining if there is an overlap of the estimated completion time of the first backup job and a start time of a second backup job subsequent to the first backup job, and if so, rescheduling the second backup job to start after the estimated completion time of the first backup job.
17. The system of claim 16 further comprising a dynamic recalculation process periodically checking for throughput changes, performing a dynamic recalculation to modify the estimated completion time using a throughput parameter, and re-estimating the completion time based on any change to the throughput parameter.
18. The system of claim 17 the throughput parameter comprises a throughput to target device parameter.
19. The system of claim 18 wherein the statistics further comprise at least one of: type of the target device, network transfer rate, processor usage during the first backup, and an amount of data to be written for the first backup.