Change-based restore from a cloud-based data protection service
A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).
1. A computer-implemented method comprising:
by a data protection service that executes on one or more cloud computing resources of a cloud, receiving a first request to restore a backup data set to one or more storage devices that comprise data generated by one or more computer systems,
wherein the one or more cloud computing resources are implemented on one or more hardware processors, and
wherein the one or more computer systems operate outside of the cloud;
by the data protection service, responsive to the first request:
submitting, to a backup agent, a change tracking query in reference to the data at the one or more storage devices, wherein the change tracking query comprises a time corresponding to the backup data set,
wherein the backup agent executes on the one or more computer systems that operate outside of the cloud,
receiving, from the backup agent, responsive to the change tracking query, results that indicate a set of changed data blocks that changed at the one or more storage devices after the time corresponding to the backup data set, and
causing a virtual machine instance to be instantiated in the cloud, wherein the virtual machine instance is associated with a block storage configured in the cloud, and
loading instructions into the virtual machine instance;
by the virtual machine instance in the cloud, based on the instructions:
identifying first data blocks, among a plurality of data objects of the backup data set, that correspond to the set of changed data blocks,
wherein the plurality of data objects of the backup data set are stored at an object storage in the cloud,
restoring the first data blocks, resulting in restored first data blocks, to the block storage configured in the cloud and associated with the virtual machine instance,
causing the restored first data blocks to be stored in the object storage in the cloud, and
providing to the backup agent an object identifier corresponding to each of the restored first data blocks stored in the object storage;
by the data protection service, after providing the backup agent with the object identifier corresponding to each of the restored first data blocks stored in the object storage, deleting the virtual machine instance and the block storage; and
by the backup agent, obtaining each of the restored first data blocks from the object storage using a corresponding object identifier, and storing each of the restored first data blocks in the one or more storage devices.
2. The computer-implemented method of claim 1 , wherein the restored first data blocks are stored in the object storage in the cloud with compression and encryption.
3. The computer-implemented method of claim 1 , wherein the backup agent is part of a local agent that executes the one or more computer systems, which operate in a data center that is distinct from the cloud, and wherein the local agent is registered with the data protection service.
4. The computer-implemented method of claim 1 , wherein the backup agent is part of a local agent that executes on the one or more computer systems that operate outside of the cloud, and wherein the local agent is configured to determine which blocks of data have changed at the one or more storage devices since a most recent backup.
5. The computer-implemented method of claim 1 , wherein the one or more computer systems operate at a data center that is distinct from the cloud.
6. The computer-implemented method of claim 1 , wherein the one or more computer systems operate at a data center that is part of a second cloud that is distinct from the cloud.
7. The computer-implemented method of claim 1 , wherein the data generated by one or more computer systems is generated by a first virtual machine that executes on one of the one or more computer systems,
wherein the data is stored in one or more virtual disks that are associated with the first virtual machine, and
wherein the one or more virtual disks are configured in the one or more storage devices; and
wherein, after the restored first data blocks have been stored in the one or more storage devices, accessing the restored first data blocks, by the first virtual machine, at the one or more virtual disks configured in the one or more storage devices.
8. The computer-implemented method of claim 1 , wherein the data generated by one or more computer systems is generated by a first database that executes on one of the one or more computer systems; and
wherein, after the restored first data blocks have been stored in the one or more storage devices, accessing the restored first data blocks, by the first database, at the one or more storage devices.
9. The computer-implemented method of claim 1 , wherein the data generated by one or more computer systems is generated by a first file system that executes on one of the one or more computer systems; and
wherein, after the restored first data blocks have been stored in the one or more storage devices, accessing the restored first data blocks, by the first file system, at the one or more storage devices.
10. The computer-implemented method of claim 1 , wherein the results responsive to the change tracking query further indicate which data blocks did not change at the one or more storage devices after the time corresponding to the backup data set.
11. A computer-implemented method comprising:
by a data protection service that executes on one or more cloud computing resources of a cloud, receiving a first request to restore a backup data set to one or more storage devices that comprise data generated by one or more computer systems,
wherein the one or more cloud computing resources are implemented on one or more hardware processors, and
wherein the one or more computer systems operate outside of the cloud;
by the data protection service, responsive to the first request:
submitting, to a backup agent, a change tracking query in reference to the data at the one or more storage devices, wherein the change tracking query comprises a time corresponding to the backup data set,
wherein the backup agent executes on the one or more computer systems that operate outside of the cloud,
receiving, from the backup agent, responsive to the change tracking query, results that indicate a set of changed data blocks that changed at the one or more storage devices after the time corresponding to the backup data set, and
causing a virtual machine instance to be instantiated in the cloud, wherein the virtual machine instance is associated with a block storage configured in the cloud, and
loading instructions into the virtual machine instance;
by the virtual machine instance in the cloud, based on the instructions:
identifying first data blocks, among a plurality of data objects of the backup data set, that correspond to the set of changed data blocks,
wherein the plurality of data objects of the backup data set are stored at an object storage in the cloud,
restoring the first data blocks, resulting in restored first data blocks, to the block storage configured in the cloud and associated with the virtual machine instance,
causing the restored first data blocks to be stored in the object storage in the cloud,
concatenating the restored first data blocks into a plurality of second data blocks, and
transmitting the plurality of second data blocks, which are compressed and encrypted, to the object storage;
by the data protection service, after providing the backup agent with object identifiers corresponding to the plurality of second data blocks stored in the object storage, deleting the virtual machine instance and the block storage;
by the backup agent, obtaining the plurality of second data blocks from the object storage using the corresponding object identifiers; and
by the backup agent, decrypting and decompressing the plurality of second data blocks, resulting in the restored first data blocks, which are stored in the one or more storage devices at respective offsets that are based on offset information received in the results responsive to the change tracking query.
12. The computer-implemented method of claim 11 , wherein the backup agent is part of a local agent that executes the one or more computer systems, which operate in a data center that is distinct from the cloud, and wherein the local agent is registered with the data protection service.
13. The computer-implemented method of claim 11 , wherein the backup agent is part of a local agent that executes on the one or more computer systems that operate outside of the cloud, and wherein the local agent is configured to determine which blocks of data have changed at the one or more storage devices since a most recent backup.
14. The computer-implemented method of claim 11 , wherein the one or more computer systems operate at a data center that is distinct from the cloud.
15. The computer-implemented method of claim 11 , wherein the one or more computer systems operate at a data center that is part of a second cloud that is distinct from the cloud.
16. The computer-implemented method of claim 11 , wherein the data generated by one or more computer systems is generated by a first virtual machine that executes on one of the one or more computer systems,
wherein the data is stored in one or more virtual disks that are associated with the first virtual machine, and
wherein the one or more virtual disks are configured in the one or more storage devices; and
wherein, after the restored first data blocks have been stored in the one or more storage devices, accessing the restored first data blocks, by the first virtual machine, at the one or more virtual disks configured in the one or more storage devices.
17. The computer-implemented method of claim 11 , wherein the data generated by one or more computer systems is generated by a first database that executes on one of the one or more computer systems; and
wherein, after the restored first data blocks have been stored in the one or more storage devices, accessing the restored first data blocks, by the first database, at the one or more storage devices.
18. The computer-implemented method of claim 11 , wherein the data generated by one or more computer systems is generated by a first file system that executes on one of the one or more computer systems; and
wherein, after the restored first data blocks have been stored in the one or more storage devices, accessing the restored first data blocks, by the first file system, at the one or more storage devices.
19. The computer-implemented method of claim 11 , wherein the results responsive to the change tracking query further indicate which data blocks did not change at the one or more storage devices after the time corresponding to the backup data set.