Systems and methods for continuous data protection with near zero recovery point
Example embodiments relate generally to systems and methods for continuous data protection (CDP) and more specifically to an input and output (I/O) filtering framework and log management system to seek a near-zero recovery point objective (RPO).
1. A method for continuous data protection for a virtual machine (VM) having a virtual disk, the method comprising operations comprising:
establishing an instance of a log replication sender at a backup site to communicate with a log replication receiver at a replication target;
capturing a base snapshot of the virtual disk;
receiving, at the backup site, I/O data from an intercepted I/O stream between the VM and a virtualization server;
buffering the received I/O data into memory and flushing the I/O data to a first log file in a first plurality of log files at a log receiver, each log file in the first plurality of log files including I/O data that was buffered from the intercepted I/O stream between the VM and the virtualization server, the first plurality of log files including a second plurality of log files, the first plurality of log files, at the log receiver, including a second log file with the base snapshot in an I/O log to form a recoverable snapshot-log chain;
determining a request for recoverable data from the replication target;
pushing the requested data to the replication target based at least on a portion of the recoverable snapshot-log chain; and
removing the second plurality of log files from the recoverable snapshot-log chain based on a first service level agreement policy, the second plurality of log files including the second log file.
2. The method of claim 1 , wherein the operations further comprise configuring the log replication receiver at the replication target to run at an unmodified existing snapshot recovery service.
3. The method of claim 1 , wherein the operations further comprise configuring a CDP metadata service to communicate with the log replication sender and the log replication receiver.
4. The method of claim 3 , wherein the operations further comprise configuring the log replication sender to identify a log tile to be replicated in the first plurality of log files at the log receiver and, based on the identified log file including a closed file, including contents of the log file in a closed replication.
5. The method of claim 3 , wherein the operations further comprise configuring the log replication sender to identify a log file to be replicated in the first plurality of log files at the log receiver and, based on the identified log file including an open file receiving I/O data from the intercepted I/O stream, including the I/O data in a live replication.
6. A system for continuous data protection for a virtual machine (VM) having a virtual disk, the system comprising:
at least one processor for executing machine-readable instructions; and
a memory storing instructions configured to cause the at least one processor to perform operations comprising:
establishing an instance of a log replication sender at a backup site to communicate with a log replication receiver at a replication target;
capturing a base snapshot of the virtual disk;
receiving, at the backup site, I/O data from an intercepted I/O stream between the VM and a virtualization server;
buffering the received I/O data into memory and flushing the I/O data to a first log file in a first plurality of log files at a log receiver, each log file in the first plurality of log files including I/O data that was buffered from the intercepted I/O stream between the VM and the virtualization server, the first plurality of log files including a second plurality of log files, the first plurality of log files, at the log receiver, including a second log file with the base snapshot in an I/O log to form a recoverable snapshot-log chain;
determining a request for recoverable data from the replication target;
pushing the requested data to the replication target based at least on a portion of the recoverable snapshot-log chain; and
removing the second plurality of log files from the recoverable snapshot-log chain based on a first service level agreement policy, the second plurality of log files including the second log file.
7. The system of claim 6 , wherein the operations further comprise configuring the log replication receiver at the replication target to run at an unmodified existing snapshot recovery service.
8. The system of claim 6 , wherein the operations further comprise configuring a CDP metadata service to communicate with the log replication sender and the log replication receiver.
9. The system of claim 8 , wherein the operations further comprise configuring the log replication sender to identify a log file to be replicated in the first plurality of log files at the log receiver and, based on the identified log file including a closed file, including contents of the log file in a closed replication.
10. The system of claim 8 , wherein the operations further comprise configuring the log replication sender to identify a log file to be replicated in the first plurality of log files at the log receiver and, based on the identified log file including an open file receiving I/O data from the intercepted I/O stream, including the I/O data in a live replication.
11. A non-transitory, machine-readable medium storing instructions which, when read by a machine, cause the machine to perform operations in a method for continuous data protection for a virtual machine (VM) having a virtual disk, the operations comprising:
establishing an instance of a log replication sender at a backup site to communicate with a log replication receiver at a replication target;
capturing a base snapshot of the virtual disk;
receiving, at the backup site, I/O data from an intercepted I/O stream between the VM and a virtualization server;
buffering the received I/O data into memory and flushing the I/O data to a first log file in a first plurality of log files at a log receiver, each log file in the first plurality of log files including I/O data that was buffered from the intercepted I/O stream between the VM and the virtualization server, the first plurality of log files including a second plurality of log files, the first plurality of log files, at the log receiver, including a second log file with the base snapshot in an I/O log to form a recoverable snapshot-log chain;
determining a request for recoverable data from the replication target;
pushing the requested data to the replication target based at least on a portion of the recoverable snapshot-log chain; and
removing the second plurality of log files from the recoverable snapshot-log chain based on a first service level agreement policy, the second plurality of log files including the second log file.
12. The medium of claim 11 , wherein the operations further comprise configuring the log replication receiver at the replication target to run at an unmodified existing snapshot recovery service.
13. The medium of claim 11 , wherein the operations further comprise configuring a CDP metadata service to communicate with the log replication sender and the log replication receiver.
14. The medium of claim 13 , wherein the operations further comprise configuring the log replication sender to identify a log file to be replicated in the first plurality of log files at the log receiver and, based on the identified log file including a closed file, including contents of the log file in a closed replication.
15. The medium of claim 14 , wherein the operations further comprise configuring the log replication sender to identify a log file to be replicated in the first plurality of log files at the log receiver and, based on the identified log file including an open file receiving I/O data from the intercepted I/O stream, including the I/O data in a live replication.