Systems and methods for change block tracking for backing up changed data
Aspects of the present disclosure enable data protection operations including differential and incremental backups by performing changed-block tracking in network or cloud computing systems with architectures that do not natively support changed-block tracking or do not expose changed-block tracking functionality to an information management system. In certain aspects, an identity of changed blocks may be obtained by using a hypervisor configured to interface with the cloud computing architecture. The identified changed blocks may be used to generate a map of the changed blocks. The maps of the changed blocks can be used by a virtual server agent to extract the changed blocks from a copy of a virtual machine disk and backed up to perform a differential or incremental backup.
1. A computer-implemented method of performing a backup of changed data associated with a virtual machine client implemented in a virtualized computing environment, the computer-implemented method comprising:
issuing a request to a compute service to provide a listing of changed data associated with a data set involved in a backup operation, the data set corresponding to primary data generated by the virtual machine client and stored in a primary storage device,
wherein the request is issued using an application programming interface (API),
wherein the virtual machine client is executed by a virtual machine monitor (VMM),
wherein the compute service:
causes the VMM to generate an image of a volume of data corresponding to the data set, and
generates a map of changed blocks based, at least in part, on the image of the volume;
receiving the map from the compute service;
using the map, extracting from the primary storage device, backup data corresponding to the changed blocks; and
forwarding the extracted backup data for storage in a secondary storage device.
2. The computer-implemented method of claim 1 , wherein the image of the volume comprises a point-in-time image.
3. The computer-implemented method of claim 1 , wherein the compute service: obtains an address of a first changed block and a number of consecutive changed blocks beginning from the address of the first changed block.
4. The computer-implemented method of claim 1 , wherein the compute service: causes an image generation command is a QEMU Monitor Protocol (QMP) command and wherein the image is in a QEMU copy on write (QCOW2) format.
5. The computer-implemented method of claim 1 , further comprising storing one or more of the image of the volume or the map of the changed blocks at a staging repository.
6. The computer-implemented method of claim 5 , wherein the staging repository is configured to store a plurality of images of the volume to use in generating changed block maps.
7. The computer-implemented method of claim 5 , wherein extracting the changed blocks comprises loading the map from the staging repository and using the map to identify data to copy from the primary storage device.
8. The computer-implemented method of claim 1 , wherein the image of the volume is in a first format, and the volume stored on the primary storage device is in a second format.
9. The computer-implemented method of claim 1 , wherein the secondary storage device resides in a secondary storage subsystem that does not form a part of the virtualized computing environment hosted by a virtualization manager.
10. The computer-implemented method of claim 9 , wherein the VMM, and the virtualization manager are implemented in a hosted cloud environment.
11. The computer-implemented method of claim 1 , further comprising: loading a snapshot of the volume as part of extracting the backup data from the primary storage device.
12. The computer-implemented method of claim 1 , wherein the changed blocks comprise an address of the changed blocks within the volume.
13. A computer-implemented system of performing a backup of changed data associated with a virtual machine client implemented in a virtualized computing environment, the computer-implemented system comprising:
a compute service node configured to:
receive a request from a data agent to provide a listing of changed data associated with a data set involved in a backup operation, the data set corresponding to primary data generated by the virtual machine client and stored in a primary storage device,
wherein the virtual machine client is executed by a virtual machine monitor (VMM), and
wherein the compute service node provides an application programming interface between the data agent and the compute service node;
cause the VMM to generate an image of a volume of data corresponding to the data set;
identify, based at least on the image of the volume, at least one or more changed blocks;
generate a map of the at least one or more changed blocks at least in part on the identified one or more changed blocks; and
transmit the map to the data agent,
wherein the data agent uses the map to extract, from the primary storage device, backup data corresponding to the changed blocks, and forwards the extracted backup data for storage in a secondary storage device.
14. The computer-implemented system of claim 13 , wherein the compute service node is further configured to: cause an image generation command is a QEMU Monitor Protocol (QMP) command and wherein the image is in a QEMU copy on write (QCOW2) format.
15. The computer-implemented system of claim 13 , wherein the compute service node is further configured to: store one or more of the image of the volume or the map of the at least one or more changed blocks at a staging repository.
16. The computer-implemented system of claim 15 , wherein the staging repository is configured to store a plurality of images of the volume to use in generating changed block maps.
17. The computer-implemented system of claim 16 , wherein the compute service node is further configured to extract the changed blocks comprises loading the map from the staging repository and using the map to identify data to copy from the primary storage device.
18. The computer-implemented system of claim 13 , wherein the image of the volume is in a first format, and the volume stored on the primary storage device is in a second format.
19. The computer-implemented system of claim 13 , wherein the secondary storage device resides in a secondary storage subsystem that does not form a part of the virtualized computing environment hosted by a virtualization manager.
20. The computer-implemented system of claim 13 , wherein the data agent loads a snapshot of the volume as part of extracting the backup data from the primary storage device.