Cloud-based destination for block-level data replication processing
Systems and methods disclosed herein improve on current technology for block-level data replication to cloud computing environments. A new system architecture deploys one or more replication tail proxies in a cloud computing environment, locally (at the cloud) tracks replicated data and determines which replicated data meet criteria for reconstructing a desired point-in-time in the cloud, and persists data blocks received at the replication tail proxy until they are processed as recovery points. The disclosed approach presents resiliency and performance advantages. First, the resiliency of block-level data replication to cloud is improved by deploying the replication tail proxy in the destination cloud. Second, a Recovery Time Objective (RTO) is reduced by enabling faster cloud deployment of virtual machines for disaster recovery, failover, and/or test purposes based on the replicated data.
1 . A system comprising:
a first computing device, which operates in a cloud computing environment and comprises one or more hardware processors, wherein the first computing device is configured to:
receive changed data blocks of a first virtual disk from a second computing device that operates outside of the cloud computing environment,
wherein the first virtual disk is associated with a first virtual machine that is hosted by a host computing device that operates outside the cloud computing environment, and
wherein the second computing device comprises one or more hardware processors,
store the changed data blocks at a queue maintained by the first computing device,
in a first-in-first-out order, process the changed data blocks from the queue into data objects that are stored in a first cloud journal, wherein the first cloud journal is associated with the first virtual machine,
based on a first marker that indicates a first point-in-time, associate the first cloud journal with the first point-in-time,
transmit the first cloud journal to cloud object storage in the cloud computing environment, and
after the first marker, process one or more second changed data blocks in the queue into second data objects that are stored at least one second cloud journal that is distinct from the first cloud journal; and
wherein the first computing device, responsive to a request to use a second virtual machine in the cloud computing environment, wherein the request indicates the first virtual machine and the first point-in-time, is further configured to:
identify the first cloud journal based on its association with the first virtual machine and with the first point-in-time,
cause data objects from the first cloud journal in the cloud object storage to be transmitted to a second virtual disk configured in the cloud computing environment,
cause the second virtual disk to attach the second virtual machine, and
cause the second virtual machine to be activated in the cloud computing environment, wherein in the second virtual disk, the second virtual machine gains access to data of the first virtual machine as of the first point-in-time.
2 . The system of claim 1 , wherein the first marker is received among the changed data blocks.
3 . The system of claim 1 , wherein the first marker is received separately from the changed data blocks.
4 . The system of claim 1 , further comprising: storing the first cloud journal at the first computing device until receiving the first marker from the second computing device.
5 . The system of claim 1 , further comprising: storing the first cloud journal at the first computing device until detecting the first marker in the queue.
6 . The system of claim 1 , wherein storing the changed data blocks at the queue enables the first computing device to respond to an other request to use a third virtual machine in the cloud computing environment before the first cloud journal is transmitted to the cloud object storage, by restoring at least one changed data block among the changed data blocks in the queue to a third virtual disk, which is associated with the third virtual machine and is configured in the cloud computing environment.
7 . The system of claim 1 , wherein the second virtual machine is activated in the cloud computing environment after all data objects from the first cloud journal have been transmitted to the second virtual disk.
8 . The system of claim 1 , further comprising: based on the first marker, closing the first cloud journal before transmitting the first cloud journal to the cloud object storage in the cloud computing environment.
9 . The system of claim 1 , wherein receiving the changed data blocks of the first virtual disk from the second computing device is based on block-level data replication from the second computing device to the first computing device.
10 . The system of claim 1 , wherein the cloud object storage comprises a plurality of recovery point stores,
wherein each recovery point store among the plurality of recovery point stores is associated with a distinct virtual machine that operates outside the cloud computing environment, and
wherein a first recovery point store among the plurality of recovery point stores comprises cloud journals that are associated with the first virtual machine, including the first cloud journal.
11 . The system of claim 10 , wherein the first virtual machine and the first recovery point store form a replication pair.
12 . A computer-implemented method comprising:
by a first computing device, which operates in a cloud computing environment and comprises one or more hardware processors:
receiving changed data blocks of a first virtual disk from a second computing device that operates outside of the cloud computing environment,
wherein the first virtual disk is associated with a first virtual machine that is hosted by a host computing device that operates outside the cloud computing environment, and
wherein the second computing device comprises one or more hardware processors;
storing the changed data blocks at a queue maintained by the first computing device;
in a first-in-first-out order, processing the changed data blocks from the queue into data objects that are stored in a first cloud journal,
wherein the first cloud journal is associated with the first virtual machine;
based on a first marker that indicates a first point-in-time, associating the first cloud journal with the first point-in-time;
transmitting the first cloud journal to cloud object storage in the cloud computing environment;
after the first marker, processing one or more second changed data blocks in the queue into second data objects that are stored in at least one second cloud journal that is distinct from the first cloud journal; and
responsive to a request to use a second virtual machine in the cloud computing environment, wherein the request indicates the first virtual machine and the first point-in-time:
identifying the first cloud journal based on its association with the first virtual machine and with the first point-in-time,
causing data objects from the first cloud journal in the cloud object storage to be transmitted to a second virtual disk configured in the cloud computing environment,
attaching the second virtual disk to the second virtual machine, and
activating the second virtual machine in the cloud computing environment.
13 . The computer-implemented method of claim 12 , wherein the first marker is received among the changed data blocks.
14 . The computer-implemented method of claim 12 , wherein the first marker is received separately from the changed data blocks.
15 . The computer-implemented method of claim 12 , further comprising: storing the first cloud journal at the first computing device until receiving the first marker from the second computing device.
16 . The computer-implemented method of claim 12 , further comprising: storing the first cloud journal at the first computing device until detecting the first marker in the queue.
17 . The computer-implemented method of claim 12 , further comprising: based on the first marker, closing the first cloud journal before transmitting the first cloud journal to the cloud object storage in the cloud computing environment, and
wherein the second virtual machine is activated in the cloud computing environment after all data objects from the first cloud journal have been transmitted to the second virtual disk.
18 . The computer-implemented method of claim 12 , wherein receiving the changed data blocks of the first virtual disk from the second computing device is based on block-level data replication from the second computing device to the first computing device.
19 . The computer-implemented method of claim 12 , wherein the cloud object storage comprises a plurality of recovery point stores,
wherein each recovery point store among the plurality of recovery point stores is associated with a distinct virtual machine that operates outside the cloud computing environment, and
wherein a first recovery point store among the plurality of recovery point stores comprises cloud journals that are associated with the first virtual machine, including the first cloud journal.
20 . The computer-implemented method of claim 19 , wherein the first virtual machine and the first recovery point store form a replication pair.