Snapshot replication operations based on incremental block change tracking
A system according to certain aspects improves the process of performing snapshot replication operations (e.g., maintaining a mirror copy of primary data at a secondary location by generating snapshots of the primary data). The system can collect and maintain cumulative block-level changes to the primary data after each sub-interval of a plurality of sub-intervals between the snapshots. When a snapshot is generated, any changes to the primary data not reflected in the cumulative block-level changes are identified based on the snapshot and transmitted to the secondary location along with the cumulative block-level changes. By the time the snapshot is generated, some or all of the changes to the primary data associated with the given snapshot have already been included in the cumulative block-level changes, thereby reducing the time and computing resources spent to identify and collect the changes for transmission to the secondary location.
1. A method for performing a data protection operation, the method comprising:
receiving first snapshot data associated with a first snapshot of primary data at a first time;
determining a first block-level change to the primary data that occurred between the first time and a second time subsequent to the first time;
receiving cumulative change data in an intermediate staging area based at least on the first block-level change; and
receiving second snapshot data associated with a second snapshot of the primary data at a third time subsequent to the second time, wherein the second snapshot data includes (i) the cumulative change data stored in the intermediate staging area, and (ii) additional change data reflecting a second block-level change to the primary data that occurred between the second time and the third time.
2. The method of claim 1 , further comprising determining the first block-level change based at least on a first change bitmap received from a filter driver.
3. The method of claim 2 , further comprising querying the filter driver for an additional change bitmap in response to using an amount of computing resources that is less than a threshold amount.
4. The method of claim 2 , wherein the first change bitmap indicates at least a first block in the primary data to which the first block-level change was made.
5. The method of claim 4 , wherein the cumulative change data comprises storing a copy of the first block indicated by the first change bitmap in the intermediate staging area.
6. The method of claim 2 , further comprising determining the second block-level change based at least on a second change bitmap received from the filter driver, wherein the second change bitmap indicates at least a second block in the primary data to which the second block-level change was made.
7. The method of claim 1 , further comprising:
determining that a first block to which the first block-level change was made is stored in the intermediate staging area; and
updating the first block stored in the intermediate staging area to reflect the first block-level change.
8. The method of claim 1 , wherein the intermediate staging area is remotely located from a client computing device such that copying data from the intermediate staging area to the destination device does not affect performance of the client computing device.
9. The method of claim 1 , wherein the length of time between the first time and the third time is substantially greater than the length of time between the second time and the third time.
10. The method of claim 1 , wherein the second snapshot includes all block change data needed to bring the primary data from the state of the primary data at the first time to the state of the primary data at the third time.
11. An apparatus that performs data protection operations, the apparatus comprising:
a destination device comprising computer hardware, the destination device configured to:
receive first snapshot data associated with a first snapshot of primary data, the first snapshot associated with a first time, wherein the first snapshot data includes at least part of the primary data;
determine a first block-level change to the primary data that occurred between the first time and a second time subsequent to the first time;
access cumulative change data stored in an intermediate staging area that is based at least on the first block-level change; and
receive second snapshot data associated with a second snapshot of the primary data, the second snapshot associated with a third time subsequent to the second time, wherein the second snapshot data includes (i) the cumulative change data stored in the intermediate staging area, and (ii) additional change data reflecting a second block-level change to the primary data that occurred between the second time and the third time.
12. The apparatus of claim 11 , wherein the first block-level change is based at least in part on a first change bitmap received from a filter driver.
13. The apparatus of claim 12 , further comprising querying the filter driver for an additional change bitmap in response to determining that use of an amount of computing resources is less than a threshold amount.
14. The apparatus of claim 12 , wherein the first change bitmap indicates at least a first block in the primary data to which the first block-level change was made.
15. The apparatus of claim 14 , wherein the cumulative change data comprises storing a copy of the first block indicated by the first change bitmap in the intermediate staging area.
16. The apparatus of claim 12 , wherein the second block-level change is based at least in part on a second change bitmap received from the filter driver, wherein the second change bitmap indicates at least a second block in the primary data to which the second block-level change was made.
17. The apparatus of claim 11 , wherein a first block to which the first block-level change was made is stored in the intermediate staging area, and a first block stored in the intermediate staging area is updated to reflect the first block-level change.
18. The apparatus of claim 11 , wherein the intermediate staging area is remotely located from a client computing device such that copying data from the intermediate staging area to the destination device does not affect performance of the client computing device.
19. The apparatus of claim 11 , wherein the length of time between the first time and the third time is substantially greater than the length of time between the second time and the third time.
20. The apparatus of claim 11 , wherein the second snapshot data includes block change data needed to bring the primary data from the state of the primary data at the first time to the state of the primary data at the third time.