Block level incremental backup for QCOW2 virtual disks
Disclosed herein are various systems and methods for performing block level incremental backup operations for QCOW2 virtual disks. A first snapshot of a virtual disk is taken and used by a virtual machine at a first point in time. The virtual disk is implemented as a QCOW2 image, and taking the first snapshot causes the first snapshot to be stored in the QCOW2 image. A second snapshot of the virtual disk is taken at a second point in time. Taking the second snapshot causes the second snapshot to be stored in the QCOW2 image. One or more blocks of the virtual disk that have been modified between the first point in time and the second point in time are identified using metadata stored in the QCOW2 image. An incremental backup operation is then performed on the virtual disk by copying one or more blocks from the second snapshot to backup storage.
1. A computer-implemented method comprising:
taking a plurality of snapshots of a virtual disk used by a virtual machine, wherein
the taking the plurality of snapshots causes each snapshot of the plurality of snapshots to be stored in a Quick Emulator Copy on Write 2 (QCOW2) image;
determining whether a first snapshot of the plurality of snapshots and a second snapshot of the plurality of snapshots exist in the QCOW2 image, wherein
the first snapshot was taken at a first point in time, and
the second snapshot was taken at a second point in time; and
in response to a determination that the first snapshot and the second snapshot exist in the QCOW2 image,
identifying one or more snapshots of the plurality of snapshots, other than the first snapshot and the second snapshot, wherein
each of the one or more snapshots was taken at a corresponding point in time that is between the first point in time and the second point in time,
identifying one or more blocks of the virtual disk that have been modified between the first point in time and the second point in time, using metadata stored in the QCOW2 image, wherein
the metadata is associated with the one or more snapshots, and
performing an incremental backup operation on the virtual disk by copying the one or more blocks from the second snapshot to backup storage; and
in response to a determination that at least one of the first snapshot or the second snapshot does not exist in the QCOW2 image,
performing a full backup operation on the virtual disk.
2. The computer-implemented method of claim 1 , wherein the identifying the one or more snapshots comprises:
accessing a header of the QCOW2 image, wherein
the header identifies each snapshot of the plurality of snapshots.
3. The computer-implemented method of claim 1 , wherein the identifying the one or more blocks comprises:
accessing metadata stored in the first snapshot, wherein
the metadata stored in the first snapshot identifies at least one block in the virtual disk that was modified subsequent to the first point in time.
4. The computer-implemented method of claim 1 , wherein the identifying the one or more blocks comprises:
identifying a plurality of snapshots taken between the first point in time and the second point in time, using the metadata; and
accessing each of the plurality of snapshots taken between the first point in time and the second point in time, wherein
the accessing is performed by traversing the plurality of snapshots taken between the first point in time and the second point in time.
5. The computer-implemented method of claim 4 , further comprising:
taking a third snapshot of the virtual disk at a third point in time, wherein
the taking the third snapshot causes the third snapshot to be stored in the QCOW2 image;
initiating a subsequent incremental backup operation of the virtual disk at the third point in time;
determining whether the second snapshot has been deleted; and
in response to a determination that the second snapshot has been deleted,
aborting the subsequent incremental backup operation, and
performing a full backup operation.
6. The computer-implemented method of claim 1 , comprising causing a snapshot of the QCOW2 image to be taken at the first point in time, wherein
the identifying the one or more snapshots comprises
accessing the snapshot of the QCOW2 image, if the first snapshot has been deleted before the second point in time.
7. A non-transitory computer readable storage medium storing program instructions executable to:
take a plurality of snapshots of a virtual disk used by a virtual machine, wherein
the plurality of snapshots being taken causes each snapshot of the plurality of snapshots to be stored in a Quick Emulator Copy on Write 2 (QCOW2) image;
determine whether a first snapshot of the plurality of snapshots and a second snapshot of the plurality of snapshots exist in the QCOW2 image, wherein
the first snapshot was taken at a first point in time, and
the second snapshot was taken at a second point in time; and
in response to a determination that the first snapshot and the second snapshot exist in the QCOW2 image,
identify one or more snapshots of the plurality of snapshots, other than the first snapshot and the second snapshot, wherein
each of the one or more snapshots was taken at a corresponding point in time that is between the first point in time and the second point in time,
identify one or more blocks of the virtual disk that have been modified between the first point in time and the second point in time, using metadata stored in the QCOW2 image, wherein
the metadata is associated with the one or more snapshots, and
perform an incremental backup operation on the virtual disk by copying the one or more blocks from the second snapshot to backup storage; and
in response to a determination that at least one of the first snapshot or the second snapshot does not exist in the QCOW2 image,
perform a full backup operation on the virtual disk.
8. The non-transitory computer readable storage medium of claim 7 , wherein the program instructions executable to identify the one or more snapshots comprise further program instructions executable to:
access a header of the QCOW2 image, wherein
the header identifies each snapshot of the plurality of snapshots.
9. The non-transitory computer readable storage medium of claim 7 , wherein the program instructions executable to identify the one or more blocks comprise further program instructions executable to:
access metadata stored in the first snapshot, wherein
the metadata stored in the first snapshot identifies at least one block in the virtual disk that was modified subsequent to the first point in time.
10. The non-transitory computer readable storage medium of claim 7 , wherein the program instructions executable to identify the one or more blocks comprise further program instructions executable to:
identify a plurality of snapshots taken between the first point in time and the second point in time, using the metadata; and
access each of the plurality of snapshots taken between the first point in time and the second point in time, wherein
the each of the plurality of snapshots taken between the first point in time and the second point in time are accessed by traversing the plurality of snapshots taken between the first point in time and the second point in time.
11. The non-transitory computer readable storage medium of claim 10 , wherein the program instructions further comprise program instructions executable to:
take a third snapshot of the virtual disk at a third point in time, wherein
the third snapshot being taken causes the third snapshot to be stored in the QCOW2 image; and
initiating a subsequent incremental backup operation of the virtual disk at the third point in time.
12. The non-transitory computer readable storage medium of claim 11 , wherein the program instructions further comprise program instructions executable to:
determine whether the second snapshot has been deleted; and
in response to a determination that the second snapshot has been deleted,
abort the subsequent incremental backup operation, and
perform a full backup operation.
13. The non-transitory computer readable storage medium of claim 7 , wherein the program instructions further comprise program instructions executable to:
cause a snapshot of the QCOW2 image to be taken at the first point in time, wherein
the program instructions executable to identify the one or more snapshots comprise program instructions executable to
access the snapshot of the QCOW2 image, if the first snapshot has been deleted before the second point in time.
14. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to
take a plurality of snapshots of a virtual disk used by a virtual machine, wherein
the plurality of snapshots being taken causes each snapshot of the plurality of snapshots to be stored in a Quick Emulator Copy on Write 2 (QCOW2) image,
determine whether a first snapshot of the plurality of snapshots and a second snapshot of the plurality of snapshots exist in the QCOW2 image, wherein
the first snapshot was taken at a first point in time, and
the second snapshot was taken at a second point in time, and
in response to a determination that the first snapshot and the second snapshot exist in the QCOW2 image,
identify one or more snapshots of the plurality of snapshots, other than the first snapshot and the second snapshot, wherein
each of the one or more snapshots was taken at a corresponding point in time that is between the first point in time and the second point in time,
identify one or more blocks of the virtual disk that have been modified between the first point in time and the second point in time, using metadata stored in the QCOW2 image, wherein
the metadata is associated with the one or more snapshots, and
perform an incremental backup operation on the virtual disk by copying the one or more blocks from the second snapshot to backup storage; and
in response to a determination that at least one of the first snapshot or the second snapshot does not exist in the QCOW2 image,
perform a full backup operation on the virtual disk.
15. The system of claim 14 , wherein the program instructions executable by the one or more processors to identify the one or more snapshots comprise further program instructions executable by the one or more processors to:
access a header of the QCOW2 image, wherein
the header identifies each snapshot of the plurality of snapshots.
16. The system of claim 14 , wherein the program instructions executable by the one or more processors to identify the one or more snapshots comprise further program instructions executable by the one or more processors to:
access metadata stored in the first snapshot, wherein
the metadata stored in the first snapshot identifies at least one block in the virtual disk that was modified subsequent to the first point in time.
17. The system of claim 16 , wherein the program instructions executable by the one or more processors to identify the one or more snapshots comprise further program instructions executable by the one or more processors to:
identify a plurality of snapshots taken between the first point in time and the second point in time, using the metadata; and
access each of the plurality of snapshots taken between the first point in time and the second point in time, wherein
the each of the plurality of snapshots taken between the first point in time and the second point in time are accessed by traversing the plurality of snapshots taken between the first point in time and the second point in time.
18. The system of claim 17 , wherein the program instructions executable by the one or more processors further comprise program instructions executable by the one or more processors to:
take a third snapshot of the virtual disk at a third point in time, wherein
the third snapshot being taken causes the third snapshot to be stored in the QCOW2 image; and
initiating a subsequent incremental backup operation of the virtual disk at the third point in time.
19. The system of claim 18 , wherein the program instructions executable by the one or more processors further comprise program instructions executable by the one or more processors to:
determine whether the second snapshot has been deleted; and
in response to a determination that the second snapshot has been deleted,
abort the subsequent incremental backup operation, and
perform a full backup operation.
20. The system of claim 14 , wherein the program instructions executable by the one or more processors further comprise program instructions executable by the one or more processors to:
cause a snapshot of the QCOW2 image to be taken at the first point in time, wherein
the program instructions executable to identify the one or more snapshots comprise program instructions executable to
access the snapshot of the QCOW2 image, if the first snapshot has been deleted before the second point in time.
21. The computer-implemented method of claim 1 , wherein the determining whether the first snapshot and the second snapshot exist in the QCOW2 image comprises:
determining whether the first snapshot has been deleted from the QCOW2 image.
22. The computer-implemented method of claim 1 , wherein
the taking the first snapshot causes the first snapshot to be stored in the QCOW2 image as part of a previous full backup operation, and
the taking of the second snapshot causes the second snapshot to be stored in the QCOW2 image as part of a previous incremental backup operation.
23. The computer-implemented method of claim 1 , wherein
the full backup operation is performed without traversing a hierarchy of the QCOW2 image.
24. The computer-implemented method of claim 1 , wherein
the virtual machine is managed by a hypervisor, and
the incremental backup operation is performed without execution of a tracking driver by the virtual machine or the hypervisor.