Recovery Point Objective (RPO) driven backup scheduling in a data storage management system

1. A data storage management system comprising:

a computing device comprising one or more hardware processors, wherein the computing device is programmed to execute a storage manager, which comprises a recovery point objective (RPO) that is administered to define a maximum duration of time since a point in time representing recoverable backed up data of a dataset; and

wherein the computing device executing the storage manager is configured to:

manage backup jobs in the data storage management system,

determine a backup level for a first backup job of the dataset,

wherein the backup level is based on (i) the RPO, (ii) a backup window that includes one or more periods of time when performing the first backup job is permissible within the data storage management system, and (iii) an amount of data to be backed up from the dataset,

determine a start time for the first backup job,

wherein the start time is not pre-administered in the data storage management system, and

wherein the start time is within the backup window and sufficient to complete the first backup job within the RPO, and

at the start time, initiate the first backup job of the dataset at the backup level.

2. The system of claim 1 wherein the backup level is further based on a data retention interval for the recoverable backed up data of the dataset.

3. The system of claim 1 wherein the backup level is one of: a full backup, an incremental backup, a differential backup, and a synthetic-full backup.

4. The system of claim 1 wherein the computing device executing the storage manager is further configured to:

determine an expected duration of the first backup job at the backup level, based on a plurality of previous backup jobs for the dataset, and wherein the start time for the first backup job is further based on the expected duration.

5. The system of claim 1 wherein the computing device executing the storage manager is further configured to:

determine an expected duration of the first backup job at the backup level; and

wherein the start time is based on: the expected duration of the first backup job, respective expected durations of other pending backup jobs, and a failure history for previous backup jobs associated with the dataset.

6. The system of claim 1 wherein the start time is sufficient to allow the storage manager to re-submit one or more data objects that failed to be backed up in the first backup job to a second backup job to meet the RPO for the dataset.

7. The system of claim 1 wherein the computing device executing the storage manager is further configured to:

re-submit one or more data objects that failed to be backed up in the first backup job to a second backup job to be completed within the RPO, wherein the storage manager excludes from the second backup job any data objects of the dataset that were successfully backed up in the first backup job.

8. The system of claim 1 wherein the computing device executing the storage manager is further configured to:

re-submit one or more data objects that failed to be backed up in the first backup job to a second backup job to be completed within the RPO, wherein the second backup job is re-submitted at a time that is based at least in part on one or more failure conditions that caused the one or more data objects to fail to be backed up in the first backup job.

9. The system of claim 1 , wherein the computing device executing the storage manager is further configured to:

when one or more data objects fail to be backed up in the first backup job, accelerate a start time for a second backup job relative to other pending backup jobs so that the RPO for the dataset is met, wherein the second backup job backs up the one or more data objects that failed to be backed up in the first backup job and excludes any data objects of the dataset that were successfully backed up in the first backup job.

10. The system of claim 1 wherein the computing device executing the storage manager is further configured to:

when one or more data objects fail to be backed up in the first backup job, determine a priority order of pending backup jobs, giving a higher priority to a pending backup job with a higher prior-failure count over another pending backup job with a lower prior-failure count.

11. The system of claim 1 wherein the computing device executing the storage manager is further configured to:

when one or more data objects fail to be backed up in the first backup job, determine a priority order of pending backup jobs, wherein as between two pending backup jobs having a same prior-failure count, giving a higher priority to a pending backup job with a longer estimated completion time over another pending backup job with a shorter estimated completion time.

12. The system of claim 1 further comprising:

a data agent associated with an application that generates the dataset,

wherein the data agent executes on a computing device comprising one or more hardware processors;

wherein the computing device executing the data agent is configured to:

identify data objects within the dataset based on a granularity level,

process the data objects for backup in the first backup job,

track first data objects of the dataset that were successfully backed up in the first backup job,

report to the storage manager that second data objects of the dataset failed to be backed up in the first backup job, and

in a second backup job, process for backup the second data objects that failed to be backed up in the first backup job; and

wherein the computing device executing the storage manager is further configured to:

re-submit the second data objects to the second backup job to be completed within the RPO.

13. The system of claim 12 further comprising:

a media agent that executes on a computing device comprising one or more hardware processors, wherein the media agent is communicatively coupled with one or more storage devices for storing backup copies of the dataset resulting from at least the first backup job and the second backup job; and

wherein the computing device executing the media agent is configured to communicate to the data agent that the first data objects of the dataset were successfully backed up in the first backup job and that the second data objects of the dataset failed to be backed up in the first backup job.

14. A method comprising:

by a storage manager that executes on a computing device comprising one or more hardware processors, wherein the storage manager manages backup jobs in a data storage management system:

determining a backup level for a first backup job of a dataset,

wherein the backup level is based on a combination of:

a recovery point objective (RPO) that defines a maximum duration of time since a point in time representing recoverable backed up data of the dataset,

a backup window that includes one or more periods of time when performing the first backup job is permissible within the data storage management system, and

an amount of data to be backed up from the dataset;

determining a start time for the first backup job within the backup window,

wherein the start time is sufficient to complete the first backup job within the RPO and to allow the storage manager to re-submit one or more data objects that failed to be backed up in the first backup job to a second backup job to also complete within the RPO, and

wherein the start time is not pre-administered in the data storage management system; and

at the start time, initiating the first backup job of the dataset at the backup level.

15. The method of claim 14 , wherein the backup level is one of: a full backup, an incremental backup, a differential backup, and a synthetic-full backup; and

further comprising: determining by the storage manager an expected duration of the first backup job at the backup level, based on a plurality of previous backup jobs for the dataset.

16. The method of claim 14 , wherein the start time for the first backup job within the backup window is based at least in part on: an expected duration of the first backup job, respective expected durations of other pending backup jobs, and a failure history for previous backup jobs associated with the dataset.

17. The method of claim 14 further comprising:

by the storage manager submitting the second backup job at a time based at least in part on one or more failure conditions that caused the one or more data objects to fail to be backed up in the first backup job.

18. The method of claim 14 further comprising:

by the storage manager, when determining that one or more data objects failed to be backed up in the first backup job, accelerating a start time for the second backup job relative to other pending backup jobs so that the RPO for the dataset is met,

wherein the storage manager excludes from the second backup job any data objects of the dataset that were successfully backed up in the first backup job.

19. The method of claim 14 further comprising:

by a data agent associated with an application that generates the dataset, wherein the data agent executes on a computing device comprising one or more hardware processors:

processing data objects in the dataset for backup in the first backup job,

tracking first data objects of the dataset that were successfully backed up in the first backup job,

reporting to the storage manager that second data objects of the dataset failed to be backed up in the first backup job, and

in a second backup job, processing for backup the second data objects that failed to be backed up in the first backup job; and

by the storage manager, submitting the second data objects to the second backup job to be completed within the RPO.

20. The method of claim 19 , wherein the storage manager excludes from the second backup job the first data objects of the dataset that were successfully backed up in the first backup job, and wherein the storage manager accelerates a schedule for the second backup job relative to other pending backup jobs to meet the RPO for the dataset, including the first data objects and the second data objects.