Method and system for multi-pronged backup using real-time attributes
A method and system for backup processes that includes identifying a target volume and identifying a number of available threads to back up the target volume. The elements in the target volume are distributed among the available threads based on a currently pending size of data in the threads. The elements are stored from each thread into a backup container, and merged from each of the backup containers into a backup volume.
1. A method for a backup process, the method comprising:
identifying a target volume;
identifying a number of available threads to back up the target volume;
distributing elements in the target volume among the available threads based on a currently pending size of data in the threads; and
initiating storage of the elements from each thread into one of a plurality of backup containers,
wherein the backup is an incremental backup and wherein at least one thread of the identified threads is allocated to exclusively perform synthesis operations associated with the backup.
2. The method of claim 1 ,
wherein the at least one thread exclusively used for synthetic operations transmits information associated with unchanged elements in the target volume.
3. The method of claim 1 , wherein distributing elements in the target volume among the available threads based on the currently pending size of data in the threads comprises:
obtaining a currently pending size of data awaiting transfer in each thread; and
distributing an element in the target volume to the thread with lowest currently pending size.
4. The method of claim 1 , wherein the currently pending size of data in the threads is obtained periodically by monitoring the threads.
5. The method of claim 1 , wherein identifying the number of available threads to back up the target volume comprises:
determining the amount of data to be backed up in the target volume; and
identifying the number of available threads to back up the target volume based on the amount of data.
6. The method of claim 1 , wherein each thread is associated with one of the plurality of backup containers and wherein each of the plurality of backup containers is not associated with more than one of the threads.
7. A system for performing a backup, the system comprising:
a backup agent comprising a processor configured to:
identifying a target volume;
identifying a number of available threads to back up the target volume;
distributing elements in the target volume among the available threads based on a currently pending size of data in the threads;
transferring the elements using the available threads;
a backup storage device comprising a processor configured to:
receive the elements using the available threads;
storing the elements from each thread into one of plurality of backup containers; and
merging, after the storing, the data from each of the backup containers into a backup volume,
wherein the backup is an incremental backup and wherein at least one thread of the identified threads is allocated to exclusively perform synthesis operations associated with the backup.
8. The system of claim 7 ,
wherein the at least one thread exclusively used for synthetic operations transmits information associated with unchanged elements in the target volume.
9. The system of claim 7 , wherein distributing elements in the target volume among the available threads based on the currently pending size of data in the threads comprises:
determining a currently pending size of data awaiting transfer in each thread; and
distributing an element in the target volume to the thread with lowest currently pending size.
10. The system of claim 7 , wherein merging data from each of the backup containers into a backup volume comprises:
storing the data from each backup container into the backup volume in the same organization as the data in target volume; and
updating metadata information of the data to reflect the storing of the data in the backup volume.
11. The system of claim 7 , wherein identifying the number of available threads to back up the target volume comprises:
determining the amount of data to be backed up in the target volume; and
identifying the number of available threads to back up the target volume based on the amount of data.
12. The system of claim 7 , wherein the currently pending size of data in the threads is obtained periodically by monitoring the threads.
13. The system of claim 7 , wherein each thread is associated with one of the plurality of backup containers and wherein each of the plurality of backup containers is not associated with more than one of the threads.
14. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method, the method comprising:
identifying a target volume;
identifying a number of available threads to back up the target volume;
distributing elements in the target volume among the available threads based on a currently pending size of data in the threads; and
initiating the storage of the elements from each thread into one of a plurality of backup container,
wherein the backup is an incremental backup and wherein at least one thread of the identified threads is allocated to exclusively perform synthesis operations associated with the backup.
15. The non-transitory computer readable medium of claim 14 , the method further comprising:
wherein the at least one thread exclusively used for synthetic operations transmits information associated with unchanged elements in the target volume.
16. The non-transitory computer readable medium of claim 14 , wherein distributing elements in the target volume among the available threads based on the currently pending size of data in the threads comprises:
obtaining a currently pending size of data awaiting transfer in each thread; and
distributing an element in the target volume to the thread with lowest currently pending size.
17. The non-transitory computer readable medium of claim 14 , wherein the currently pending size of data in the threads is obtained periodically by monitoring the threads.
18. The non-transitory computer readable medium of claim 14 , wherein identifying the number of available threads to back up the target volume comprises:
determining the amount of data to be backed up in the target volume; and
identifying the number of available threads to back up the target volume based on the amount of data.