Block-based backups for large-scale volumes
View Patent ↗Embodiments are directed to a method of performing block based backups for large-scale source volumes, by obtaining a total source volume size to determine if the source volume exceeds 64 TB, creating a number of sub-volumes to divide the total source volume into discrete sub-volumes of 64 TB each, grouping the discrete sub-volumes into a container, and storing, upon a read operation for a block based backup operation, each block of the source volume into a corresponding sub-volume of the sub-volumes. The method determines a block number and offset of each block and derives the corresponding sub-volume from the offset. Each sub-volume represents an individual saveset of the backup operation, and the container manages the individual saveset for the block based backup operation.
1. A computer-implemented method of performing block based backups for large-scale source volumes, comprising:
obtaining a total source volume size to determine if the source volume exceeds 64 TB;
creating a number of sub-volumes to divide the total source volume into discrete sub-volumes of 64 TB each;
grouping the discrete sub-volumes into a container; and
storing, upon a read operation for a block based backup operation, each block of the source volume into a corresponding sub-volume of the sub-volumes.
2. The method of claim 1 further comprising determining a block number and offset of each block and determining the corresponding sub-volume from the offset.
3. The method of claim 2 wherein each sub-volume represents an individual saveset of the backup operation, and wherein the container manages the individual saveset for the block based backup operation.
4. The method of claim 1 wherein the block based backups are part of a data domain backup system.
5. The method of claim 4 wherein the source volume comprises virtual hard disk files formatted as a Virtual Hard Disk Image (VHD).
6. The method of claim 5 wherein the backup operation comprises one of a full backup, and incremental backup, and a virtual synthetic full backup, wherein the virtual synthetic full backup comprises a previous full backup and the subsequent incremental backups to form a new full backup.
7. The method of claim 6 wherein, for incremental and virtual synthetic full backups, the method further comprises creating one virtual VHD file in each corresponding sub-volume.
8. The method of claim 1 further comprising:
performing a recovery operation after the storing of each block; and
performing a data recovery operation for the source volume by mounting all the corresponding sub-volumes together using a mount driver component.
9. The method of claim 8 wherein the mount driver component opens a storage device handle for each of the corresponding sub-volumes.
10. The method of claim 8 wherein the recovery operation further comprises:
reading all used blocks from the corresponding sub-volumes in parallel; and
writing same blocks back to a target volume.
11. The method of claim 1 wherein the source volume is of a total size of 256 TB (terabytes) and a first sub-volume of the sub-volumes extends from 0 to 64 TB, a second sub-volume of the sub-volumes extends from 65 TB to 128 TB, a third sub-volume of the sub-volumes extends from 129 TB to 192 TB, and a fourth sub-volume of the sub-volumes extends from 193 TB to 256 TB.
12. A system comprising a processor-based executable module configured perform block based backups for large-scale source volumes, comprising:
an agent process obtaining a total source volume size to determine if the source volume exceeds 64 TB;
a partitioning component creating a number of sub-volumes to divide the total source volume into discrete sub-volumes of 64 TB each;
a grouper grouping the discrete sub-volumes into a container; and
a hardware backup component storing, upon a read operation for a block based backup operation, each block of the source volume into a corresponding sub-volume of the sub-volumes.
13. The system of claim 12 wherein the partitioning component determines a block number and offset of each block and determines the corresponding sub-volume from the offset, and wherein each sub-volume represents an individual saveset of the backup operation, and wherein the container manages the individual saveset for the block based backup operation.
14. The system of claim 13 wherein the backup component comprises at least part of a data domain backup system, and wherein the source volume comprises virtual hard disk files formatted as a Virtual Hard Disk Image (VHDx).
15. The system of claim 14 wherein the backup operation comprises one of a full backup, and incremental backup, and a virtual synthetic full backup, wherein the virtual synthetic full backup comprises a previous full backup and the subsequent incremental backups to form a new full backup, and wherein, for incremental and virtual synthetic full backups, the method further comprises creating one virtual VHDx file in each corresponding sub-volume.
16. The system of claim 12 further comprising:
a recovery component performing a recovery operation after the storing of each block; and
a mount driver component performing a data recovery operation for the source volume by mounting all the corresponding sub-volumes together, wherein the mount driver component opens a storage device handle for each of the corresponding sub-volumes.
17. The system of claim 16 wherein the recovery operation comprises:
reading all used blocks from the corresponding sub-volumes in parallel; and
writing same blocks back to a target volume.
18. The system of claim 12 wherein the source volume is of a total size of 256 TB (terabytes) and a first sub-volume of the sub-volumes extends from 0 to 64 TB, a second sub-volume of the sub-volumes extends from 65 TB to 128 TB, a third sub-volume of the sub-volumes extends from 129 TB to 192 TB, and a fourth sub-volume of the sub-volumes extends from 193 TB to 256 TB.
19. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method for performing block based backups for large-scale source volumes, by:
obtaining a total source volume size to determine if the source volume exceeds 64 TB;
creating a number of sub-volumes to divide the total source volume into discrete sub-volumes of 64 TB each;
grouping the discrete sub-volumes into a container; and
storing, upon a read operation for a block based backup operation, each block of the source volume into a corresponding sub-volume of the sub-volumes.
20. The computer program product of claim 19 wherein the method further comprises determining a block number and offset of each block and determining the corresponding sub-volume from the offset, and wherein each sub-volume represents an individual saveset of the backup operation, and wherein the container manages the individual saveset for the block based backup operation.