Block-based backups for large-scale volumes and advanced file type devices

1. A computer-implemented method of performing block based backups for large-scale source volumes in a deduplication backup system, comprising:

obtaining, in a server computer of the deduplication backup system, a total source volume size to determine if the source volume exceeds 64 TB;

creating a number of sub-volumes to divide the total source volume into discrete sub-volumes of 64 TB each; and

storing in an advanced file type device (AFTD) coupled to the server, upon a read operation for a block based backup operation, each block of the source volume into a corresponding sub-volume of the sub-volumes, so as to support backup and restore operations for increasing volume sizes of backups in the deduplication backup system.

2. The method of claim 1 wherein the AFTD represents a type of Backup-To-Disk (B2D) that allows volumes to be shared among multiple devices and different storage nodes to enable read/write concurrency.

3. The method of claim 2 wherein the AFTD facilitates running concurrent backup, recovery and clone or multiple clone operations by being configured to simultaneously receive many save sessions and to simultaneously handle many recover sessions.

4. The method of claim 1 wherein the AFTD allows multiple nsrmmd (networker media multiplexor daemon) processes for a single AFTD, wherein the nsrmmd daemon comprises a storage node daemon for network save and recover media multiplexing operations.

5. The method of claim 1 further comprising:

grouping the discrete sub-volumes into a container; and

determining a block number and offset of each block and determining the corresponding sub-volume from the offset, and each sub-volume represents an individual saveset of the backup operation, and wherein the container manages the individual saveset for the block based backup operation.

6. The method of claim 5 wherein the block based backups are part of a system comprising the AFTD as a separate storage node, and wherein the system comprises a Microsoft Windows based network, and wherein the source volume comprises virtual hard disk files formatted as a Microsoft Virtual Hard Disk Image (VHD).

7. The method of claim 6 wherein the backup operation comprises one of a full backup followed by one or more incremental backups to form a new full backup.

8. The method of claim 7 further wherein the one or more incremental backups form a series of child backups for the full backup, and wherein the child backups are linked to the full backup based on a sub-volume organization based on a 64 TB block range within the total source volume size.

9. The method of claim 8 further comprising:

performing a recovery operation after the storing of each block; and

performing a data recovery operation for the source volume by mounting all the corresponding sub-volumes together using a mount driver component, wherein the mount driver component opens a storage device handle for each of the corresponding sub-volumes.

10. The method of claim 9 wherein the recovery operation further comprises merging a requested block from a parent and child sub-volume into a respective group and returning the requested block.

11. The method of claim 10 wherein the merging comprises:

defining an array of file descriptors to read from a zero entry that contains the full backup and an N−1 entry that contains a descriptor for a last incremental save of the one or more incremental saves; and

defining an array of block allocation tables (BAT) to prepare distributed stream extents wherein the zero entry contains a BAT table for the full backup and an N−1 entry includes a BAT table for the last incremental save.

12. A method of performing block based backups for large-scale source volumes in a deduplication backup system, comprising:

obtaining, in a server computer of the deduplication backup system, a total source volume size to determine if the source volume exceeds 64 TB;

creating a number of sub-volumes to divide the total source volume into discrete sub-volumes of 64 TB each;

grouping the discrete sub-volumes into a container, each sub-volume containing a full backup savesets and one or more incremental backup savesets; and

storing, upon a read operation for a block based backup operation, each block of the source volume into a corresponding sub-volume of the sub-volumes in a parent-child saveset organization for each full backup saveset and respective associated incremental backup saveset, so as to support backup and restore operations for increasing volume sizes of backups in the deduplication backup system.

13. The method of claim 12 wherein the source volume is of a total size of 256 TB (terabytes) and a first sub-volume of the sub-volumes extends from 0 to 64 TB, a second sub-volume of the sub-volumes extends from 65 TB to 128 TB, a third sub-volume of the sub-volumes extends from 129 TB to 192 TB, and a fourth sub-volume of the sub-volumes extends from 193 TB to 256 TB.

14. The method of claim 13 further comprising:

performing a recovery operation after the storing of each block; and

performing a data recovery operation for the source volume by mounting all the corresponding sub-volumes together using a mount driver component, wherein the mount driver component opens a storage device handle for each of the corresponding sub-volumes.

15. The method of claim 14 wherein the recovery operation further comprises merging a requested block from a parent and child sub-volume into a respective group and returning the requested block.

16. The method of claim 15 wherein the merging comprises:

defining an array of file descriptors to read from a zero entry that contains the full backup and an N−1 entry that contains a descriptor for a last incremental save of the one or more incremental saves; and

defining an array of block allocation tables (BAT) to prepare distributed stream extents wherein the zero entry contains a BAT table for the full backup and an N−1 entry includes a BAT table for the last incremental save.

17. The method of claim 12 wherein the backup component comprises at least part of a Microsoft Windows based network, and wherein the source volume comprises virtual hard disk files formatted as a Microsoft Virtual Hard Disk Image (VHD).

18. The method of claim 17 wherein the storing comprises storing in an advanced file type device (AFTD) upon the read operation, wherein the AFTD represents a type of Backup-To-Disk (B2D) that allows volumes to be shared among multiple devices and different storage nodes to enable read/write concurrency, and facilitates running concurrent backup, recovery and clone or multiple clone operations by being configured to simultaneously receive many save sessions and to simultaneously handle many recover sessions.

19. The method of claim 18 wherein the AFTD allows multiple nsrmmd (networker media multiplexor daemon) processes for a single AFTD, wherein the nsrmmd daemon comprises a storage node daemon for network save and recover media multiplexing operations.

20. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method for performing block based backups for large-scale source volumes in a deduplication backup system, by:

obtaining, in a server computer of the deduplication backup system, a total source volume size to determine if the source volume exceeds 64 TB;

creating a number of sub-volumes to divide the total source volume into discrete sub-volumes of 64 TB each; and

storing in an advanced file type device (AFTD) coupled to the server, upon a read operation for a block based backup operation, each block of the source volume into a corresponding sub-volume of the sub-volumes, so as to support backup and restore operations for increasing volume sizes of backups in the deduplication backup system.