Obtaining full snapshots for subsets of objects over time
A data management system (DMS) may use a staggered full snapshot rollout approach to obtain full snapshots of multiple computing objects. The DMS may identify that a quantity of the multiple objects scheduled for full backups by the DMS exceeds a threshold quantity. The DMS may initiate, at a first time, and based on the quantity of objects exceeding the threshold quantity, a first set of full snapshots corresponding to a first subset of objects included in the multiple objects. The DMS may initiate, at a second time subsequent to the first time, a second set of full snapshots corresponding to a second subset of objects included in the multiple objects. The second subset of objects may be different than the first subset of objects. The DMS may continue to obtain full snapshots of respective subsets of the multiple objects until all of the scheduled full snapshots are obtained.
1. A method, comprising:
activating, by a data management system, a first storage engine;
scheduling, by the data management system, a plurality of objects for full backups by the data management system based at least in part on activating the first storage engine, wherein scheduling the plurality of objects for full backups is further based at least in part on the first storage engine being associated with a first snapshot format that is incompatible with a second snapshot format associated with a previously activated second storage engine;
acquiring, by the data management system, a first set of full snapshots of a first batch of objects from among the plurality of objects, wherein acquisition of the first set of full snapshots is initiated at a first time based at least in part on the plurality of objects exceeding a threshold quantity as of a time of the scheduling of the plurality of objects for full backups; and
acquiring, by the data management system, a second set of full snapshots of a second batch of objects from among the plurality of objects, the second batch of objects different than and exclusive of the first batch of objects, wherein acquisition of the second set of full snapshots is initiated at a second time subsequent to the first time based at least in part on the plurality of objects exceeding the threshold quantity as of the time of the scheduling of the plurality of objects for full backups.
2. The method of claim 1 , further comprising:
determining, by the data management system and for inclusion in the first batch of objects, a first quantity of objects corresponding to a percentage of the plurality of objects.
3. The method of claim 2 , further comprising:
receiving an indication of the percentage.
4. The method of claim 2 , wherein the second batch of objects comprises a second quantity of objects that is equal to the first quantity based at least in part on the percentage.
5. The method of claim 2 , further comprising:
receiving a second indication of a second percentage of the plurality of objects, wherein the second batch of objects comprises a second quantity of objects that corresponds to the second percentage.
6. The method of claim 1 , further comprising:
monitoring, by the data management system and after initiating the acquisition of the first set of full snapshots corresponding to the first batch of objects, a status of the first set of full snapshots, wherein the status corresponds to a percentage of full snapshots of the first set of full snapshots that are pending, a percentage of full snapshots of the first set of full snapshots that have been obtained by the data management system, or both; and
determining, by the data management system and for inclusion in the second batch of objects, a quantity of objects based at least in part on the status of the first set of full snapshots.
7. The method of claim 6 , further comprising:
transmitting a report that indicates the status of the first set of full snapshots based at least in part on the monitoring; and
receiving, in response to the report, an indication of the quantity of objects included in the second batch of objects based at least in part on the status.
8. The method of claim 6 , further comprising:
monitoring, by the data management system and after initiating the acquisition of the second set of full snapshots corresponding to the second batch of objects, a second status of the second set of full snapshots, wherein the second status indicates a percentage of full snapshots of the second set of full snapshots that are pending, a percentage of full snapshots of the second set of full snapshots that have been obtained by the data management system, or both;
determining, by the data management system, a third quantity of a third batch of objects from among the plurality of objects based at least in part on the status of the first set of full snapshots and the second status of the second set of full snapshots; and
acquiring, by the data management system, a third set of full snapshots corresponding to the third batch of objects included in the plurality of objects, the third batch of objects different from and exclusive of the first batch of objects and the second batch of objects, wherein acquisition of the third set of full snapshots is initiated at a third time subsequent to the second time based at least in part on the plurality of objects exceeding the threshold quantity as of the time of the scheduling of the plurality of objects for full backups.
9. The method of claim 1 , further comprising:
determining, by the data management system, a quantity of backup jobs being executed by the data management system within a time period prior to the second time; and
determining, by the data management system and for inclusion in the second batch of objects, a quantity of objects based at least in part on the quantity of jobs being executed by the data management system.
10. The method of claim 1 , further comprising:
detecting a trigger condition associated with the data management system; and
delaying initiation of the acquisition of the second set of full snapshots based at least in part on detecting the trigger condition, wherein a time period between the first time and the second time is increased based at least in part on delaying the second time.
11. The method of claim 1 , further comprising:
selecting objects from among the plurality of objects to include in the first batch of objects based at least in part on first values of first unique identifiers associated with the first batch of objects; and
selecting objects from among the plurality of objects to include in the second batch of objects based at least in part on second values of second unique identifiers associated with the second batch of objects.
12. The method of claim 11 , further comprising:
determining a first range of unique identifier values based at least in part on a first target size of the first batch of objects and a target time period for obtaining the full backups of the plurality of objects, wherein the first batch of objects is based at least in part on the first values of the first unique identifiers associated with the first batch of objects being within the first range; and
determining a second range of unique identifier values based at least in part on a second target size of the second batch of objects and the target time period, wherein the second batch of objects is based at least in part on the second values of the second unique identifiers associated with the second batch of objects being within the second range.
13. The method of claim 1 , further comprising:
obtaining, by the data management system and using a second storage engine before activating the first storage engine, second full snapshots corresponding to the plurality of objects, wherein the second full snapshots have the second snapshot format, and wherein the second storage engine is the previously activated second storage engine.
14. The method of claim 1 , further comprising:
obtaining the first set of full snapshots, the second set of full snapshots, and other sets of full snapshots corresponding to other batches of objects included in the plurality of objects; and
acquiring, in response to obtaining full snapshots corresponding to all of the plurality of objects using the first storage engine, incremental snapshots corresponding to the plurality of objects using the first storage engine.
15. An apparatus, comprising:
at least one processor;
memory coupled with the at least one processor; and
instructions stored in the memory and executable by the at least one processor to cause the apparatus to:
activate, by a data management system, a first storage engine;
schedule, by the data management system, a plurality of objects for full backups by the data management system based at least in part on activating the first storage engine, wherein scheduling the plurality of objects for full backups is further based at least in part on the first storage engine being associated with a first snapshot format that is incompatible with a second snapshot format associated with a previously activated second storage engine;
acquire, by the data management system, a first set of full snapshots of a first batch of objects from among the plurality of objects, wherein acquisition of the first set of full snapshots is initiated at a first time based at least in part on the plurality of objects exceeding a threshold quantity as of a time of the scheduling of the plurality of objects for full backups; and
acquire, by the data management system, a second set of full snapshots of a second batch of objects from among the plurality of objects, the second batch of objects different than and exclusive of the first batch of objects, wherein acquisition of the second set of full snapshots is initiated at a second time subsequent to the first time based at least in part on the plurality of objects exceeding the threshold quantity as of the time of the scheduling of the plurality of objects for full backups.
16. The apparatus of claim 15 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
determine, by the data management system and for inclusion in the first batch of objects, a first quantity of objects corresponding to a percentage of the plurality of objects.
17. The apparatus of claim 15 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
monitor, by the data management system and after initiating the acquisition of the first set of full snapshots corresponding to the first batch of objects, a status of the first set of full snapshots, wherein the status corresponds to a percentage of full snapshots of the first set of full snapshots that are pending, a percentage of full snapshots of the first set of full snapshots that have been obtained by the data management system, or both; and
determine, by the data management system and for inclusion in the second batch of objects, a quantity of objects based at least in part on the status of the first set of full snapshots.
18. A non-transitory computer-readable medium storing code, the code comprising instructions executable by at least one processor to:
activate, by a data management system, a first storage engine;
schedule, by the data management system, a plurality of objects for full backups by the data management system based at least in part on activating the first storage engine, wherein scheduling the plurality of objects for full backups is further based at least in part on the first storage engine being associated with a first snapshot format that is incompatible with a second snapshot format associated with a previously activated second storage engine;
acquire, by the data management system, a first set of full snapshots of a first batch of objects from among the plurality of objects, wherein acquisition of the first set of full snapshots is initiated at a first time based at least in part on the plurality of objects exceeding a threshold quantity as of a time of the scheduling of the plurality of objects for full backups; and
acquire, by the data management system, a second set of full snapshots of a second batch of objects from among the plurality of objects, the second batch of objects different than and exclusive of the first batch of objects, wherein acquisition of the second set of full snapshots is initiated at a second time subsequent to the first time based at least in part on the plurality of objects exceeding the threshold quantity as of the time of the scheduling of the plurality of objects for full backups.
19. The non-transitory computer-readable medium of claim 18 , wherein the instructions are further executable by the at least one processor to:
determine, by the data management system and for inclusion in the first batch of objects, a first quantity of objects corresponding to a percentage of the plurality of objects.
20. The apparatus of claim 15 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
obtain, by the data management system and using a second storage engine before activating the first storage engine, second full snapshots corresponding to the plurality of objects, wherein the second full snapshots have the second snapshot format, and wherein the second storage engine is the previously activated second storage engine.