Maintaining backup snapshots on deduplicated storage using continuous replication
View Patent ↗A method and system for maintaining backups on deduplicated storage using continuous data replication is discussed. A backup snapshot may be initialized. Application IO's may thereafter be transmitted to the deduplicated storage and placed in a journal. A second backup snapshot may be synthesized using the initial backup snapshot and/or the application IO's.
1. A method for maintaining a backup using continuous replication, the method comprising:
initializing an initial backup snapshot of a storage device on a deduplicated storage;
intercepting, by a data protection agent, IOs comprising writes to a source storage system;
sending the IOs from the data protection agent to a first data protection appliance, the first data protection appliance comprising first, second, and third modes, wherein when the first data protection appliance is in the first mode, the first data protection appliance sends an IO to a second data protection appliance, receives an acknowledgement of the IO from the second data protection appliance, and sends the acknowledgement back to the data protection agent, wherein the data protection agent waits until receipt of the acknowledgement before sending the IO to the source storage system,
wherein when the first data protection appliance is in the second mode, the first data protection appliance sends the acknowledgement to the data protection agent upon receipt of the IO from the data protection agent and before receiving the acknowledgement of the IO from the second data protection appliance, and
wherein when the first data protection appliance is in the third mode, the first data protection appliance combines two or more IOs into an aggregate, sends the aggregate to the second data protection appliance, and sends the acknowledgement to the data protection agent upon receipt of each IO in the aggregate and before receiving the acknowledgement back from the second data protection appliance;
receiving, at the second data protection appliance, the IOs comprising writes to a source storage system, wherein the source storage system is remote from the deduplicated storage;
writing IO metadata from the IOs to a metadata journal stored on the deduplicated storage;
maintaining a first level of backup granularity by writing IO data from the IOs to a first data journal stored on the deduplicated storage;
synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, the synthesizing comprising:
creating first and second pointers in a file for the second snapshot, wherein the first pointer points to first data in the first data journal, the second pointer points to second data in the initial backup snapshot, the first data comprises data that changed after the initial backup snapshot, and the second data comprises data that did not change after the initial backup snapshot;
receiving additional IOs comprising writes to the storage device after the second snapshot is synthesized; and
maintaining a second level of backup granularity, different from the first level of backup granularity, by writing the additional IOs to a second data journal stored on the deduplicated storage, wherein during the writing the additional IOs to a second data journal, the first data journal is capable of receiving the additional IOs.
2. The method of claim 1 , wherein the IOs are application generated IOs.
3. The method of claim 2 , wherein the application IOs are written to the data journal sequentially.
4. The method of claim 3 , wherein synthesizing a second snapshot comprises synthesizing data from the sequential journal IOs and the initial snapshot into the second snapshot.
5. The method of claim 1 , further comprising:
after the synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, deleting the first data journal from the deduplicated storage while maintaining the second snapshot and the second data journal on the deduplicated storage;
synthesizing a third snapshot from at least the second data journal; and
after the synthesizing a third snapshot from at least the second data journal, maintaining both the third snapshot and the second data journal.
6. The method of claim 1 , further comprising:
after the synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, deleting the first data journal from the deduplicated storage while maintaining the second snapshot and the second data journal on the deduplicated storage.
7. A computer program product for maintaining a backup using continuous replication, the computer program product comprising a non-transitory computer readable medium encoded with computer executable program, the code enabling:
initializing an initial backup snapshot of a storage device on a deduplicated storage;
intercepting, by a data protection agent, IOs comprising writes to a source storage system;
sending the IOs from the data protection agent to a first data protection appliance, the first data protection appliance comprising first, second, and third modes, wherein when the first data protection appliance is in the first mode, the first data protection appliance sends an IO to a second data protection appliance, receives an acknowledgement of the IO from the second data protection appliance, and sends the acknowledgement back to the data protection agent, wherein the data protection agent waits until receipt of the acknowledgement before sending the IO to the source storage system,
wherein when the first data protection appliance is in the second mode, the first data protection appliance sends the acknowledgement to the data protection agent upon receipt of the IO from the data protection agent and before receiving the acknowledgement of the IO from the second data protection appliance, and
wherein when the first data protection appliance is in the third mode, the first data protection appliance combines two or more IOs into an aggregate, sends the aggregate to the second data protection appliance, and sends the acknowledgement to the data protection agent upon receipt of each IO in the aggregate and before receiving the acknowledgement back from the second data protection appliance;
receiving, at the second data protection appliance, the IOs comprising writes to a source storage system, wherein the source storage system is remote from the deduplicated storage;
writing IO metadata from the IOs to a metadata journal stored on the deduplicated storage;
maintaining a first level of backup granularity by writing IO data from the IOs to a first data journal stored on the deduplicated storage;
synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, the synthesizing comprising:
creating first and second pointers in a file for the second snapshot, wherein the first pointer points to first data in the first data journal, the second pointer points to second data in the initial backup snapshot, the first data comprises data that changed after the initial backup snapshot, and the second data comprises data that did not change after the initial backup snapshot;
receiving additional IOs comprising writes to the storage device after the second snapshot is synthesized; and
maintaining a second level of backup granularity, different from the first level of backup granularity, by writing the additional IOs to a second data journal stored on the deduplicated storage, wherein during the writing the additional IOs to a second data journal, the first data journal is capable of receiving the additional IOs.
8. The method of claim 7 , wherein the IOs are application IOs.
9. The method of claim 8 , wherein the application IOs are written to the data journal sequentially.
10. The method of claim 9 , wherein synthesizing a second snapshot comprises synthesizing data from the sequential IOs and the initial snapshot into the second snapshot.
11. The method of claim 7 , further comprising:
after the synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, deleting the first data journal from the deduplicated storage while maintaining the second snapshot and the second data journal on the deduplicated storage;
synthesizing a third snapshot from at least the second data journal; and
after the synthesizing a third snapshot from at least the second data journal, maintaining both the third snapshot and the second data journal.
12. The method of claim 7 , further comprising:
after the synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, deleting the first data journal from the deduplicated storage while maintaining the second snapshot and the second data journal on the deduplicated storage.
13. A system for maintaining a backup using continuous replication, the system comprising a computer processor configured to execute instructions comprising:
initializing an initial backup snapshot of a storage device on a deduplicated storage, wherein the deduplicated storage comprises a non-transitory computer readable storage medium;
intercepting, by a data protection agent, IOs comprising writes to a source storage system;
sending the IOs from the data protection agent to a first data protection appliance, the first data protection appliance comprising first, second, and third modes, wherein when the first data protection appliance is in the first mode, the first data protection appliance sends an IO to a second data protection appliance, receives an acknowledgement of the IO from the second data protection appliance, and sends the acknowledgement back to the data protection agent, wherein the data protection agent waits until receipt of the acknowledgement before sending the IO to the source storage system,
wherein when the first data protection appliance is in the second mode, the first data protection appliance sends the acknowledgement to the data protection agent upon receipt of the IO from the data protection agent and before receiving the acknowledgement of the IO from the second data protection appliance, and
wherein when the first data protection appliance is in the third mode, the first data protection appliance combines two or more IOs into an aggregate, sends the aggregate to the second data protection appliance, and sends the acknowledgement to the data protection agent upon receipt of each IO in the aggregate and before receiving the acknowledgement back from the second data protection appliance;
receiving, at the second data protection appliance, the IOs comprising writes to a source storage system, wherein the source storage system is remote from the deduplicated storage;
writing IO metadata from the IOs to a metadata journal stored on the deduplicated storage;
maintaining a first level of backup granularity by writing IO data from the IOs to a first data journal stored on the deduplicated storage;
synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, the synthesizing comprising:
creating first and second pointers in a file for the second snapshot, wherein the first pointer points to first data in the first data journal, the second pointer points to second data in the initial backup snapshot, the first data comprises data that changed after the initial backup snapshot, and the second data comprises data that did not change after the initial backup snapshot;
receiving additional IOs comprising writes to the storage device after the second snapshot is synthesized; and
maintaining a second level of backup granularity, different from the first level of backup granularity, by writing the additional IOs to a second data journal stored on the deduplicated storage, wherein during the writing the additional IOs to a second data journal, the first data journal is capable of receiving the additional IOs.
14. The method of claim 13 , wherein the IOs are sequential IOs.
15. The method of claim 14 , wherein the sequential IOs are written to the data journal sequentially.
16. The method of claim 15 , wherein synthesizing a second snapshot comprises synthesizing data from the sequential IOs and the initial snapshot into the second snapshot.
17. The method of claim 13 , further comprising:
after the synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, deleting the first data journal from the deduplicated storage while maintaining the second snapshot and the second data journal on the deduplicated storage;
synthesizing a third snapshot from at least the second data journal; and
after the synthesizing a third snapshot from at least the second data journal, maintaining both the third snapshot and the second data journal.
18. The method of claim 13 , further comprising:
after the synthesizing, without copying over data from the source storage system, a second snapshot from the initial backup snapshot and the first data journal, deleting the first data journal from the deduplicated storage while maintaining the second snapshot and the second data journal on the deduplicated storage.
19. The method of claim 1 comprising deleting one of the first or second data journals on the deduplicated storage to increase storage space on the deduplicated storage, and not deleting another of the first or second data journals on the deduplicated storage to allow point-in-time recoveries covered by the other of the first or second data journals that is not deleted.
20. The method of claim 1 wherein the first level of backup granularity defines a set of first point-in-time recoveries, the second level of backup granularity defines a set of second point-in-time recoveries, and the second point-in-time recoveries correspond to times after the first point-in-time recoveries.