Multi stream deduplicated backup of collaboration server data
View Patent ↗Techniques to backup collaboration server data are disclosed. A plurality of separate threads are used to back up a prescribed number of subsets of a dataset. Each subset of the prescribed number of subsets was determined by walking an associated directory in the prescribed order at a prior time with which a prior backup is associated. A separate thread is spawned to backup transaction logs associated with the dataset. A first pass backup of the transaction logs is performed in parallel with the backup of the prescribed number of subsets.
1. A method of backing up data, comprising:
using a plurality of separate threads to back up a prescribed number of subsets of a dataset, wherein each subset of the prescribed number of subsets was determined by walking an associated directory in a prescribed order at a prior time with which a prior backup is associated;
spawning a separate thread to backup transaction logs associated with the dataset; and
performing a first pass backup of the transaction logs in parallel with the backup of the prescribed number of subsets.
2. The method of claim 1 , further comprising performing a second pass backup of the transaction logs.
3. The method of claim 1 , wherein the plurality of separate threads are configured to provide data included in a corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the corresponding subset and a previous corresponding subset associated with a prior backup.
4. The method of claim 3 , wherein each de-duplicating instance uses a corresponding local cache located at a collaboration server.
5. The method of claim 1 , wherein the prescribed number of subsets corresponds to a configured maximum number of concurrent threads used to back up the subsets in parallel.
6. The method of claim 1 , further comprising generating for each subset a corresponding list of files.
7. The method of claim 6 , further comprising streaming each corresponding list of files to a corresponding thread.
8. The method of claim 1 , further comprising:
determining one or more changes made to the prescribed number of subsets while the first pass backup of the transaction logs is performed in parallel with the backup of the prescribed number of subsets; and
performing a second pass backup of the transaction logs by backing up the one or more changes.
9. A system for backing up data, comprising:
a processor configured to:
use a plurality of separate threads to back up a prescribed number of subsets of a dataset, wherein each subset of the prescribed number of subsets was determined by walking an associated directory in a prescribed order at a prior time with which a prior backup is associated;
spawn a separate thread to backup transaction logs associated with the dataset; and
perform a first pass backup of the transaction logs in parallel with the backup of the prescribed number of subsets.
10. The system of claim 9 , wherein the processor is further configured to perform a second pass backup of the transaction logs.
11. The system of claim 9 , further comprising performing a second pass backup of the transaction logs.
12. The system of claim 9 , wherein the plurality of separate threads are configured to provide data included in a corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the corresponding subset and a previous corresponding subset associated with a prior backup.
13. The system of claim 12 , wherein each de-duplicating instance uses a corresponding local cache located at a collaboration server.
14. The system of claim 9 , wherein the prescribed number of subsets corresponds to a configured maximum number of concurrent threads used to back up the subsets in parallel.
15. The system of claim 9 , further comprising generating for each subset a corresponding list of files.
16. The system of claim 15 , further comprising streaming each corresponding list of files to a corresponding thread.
17. A computer program product to back up data, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
using a plurality of separate threads to back up a prescribed number of subsets of a dataset, wherein each subset of the prescribed number of subsets was determined by walking an associated directory in a prescribed order at a prior time with which a prior backup is associated;
spawning a separate thread to backup transaction logs associated with the dataset; and
performing a first pass backup of the transaction logs in parallel with the backup of the prescribed number of subsets.
18. The method of claim 1 , further comprising performing a second pass backup of the transaction logs.
19. The method of claim 1 , wherein the plurality of separate threads are configured to provide data included in a corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the corresponding subset and a previous corresponding subset associated with a prior backup.
20. The method of claim 3 , wherein each de-duplicating instance uses a corresponding local cache located at a collaboration server.