CHANGE BLOCK TRACKING FOR TRANSFER OF DATA FOR BACKUPS
In one approach, a set of data blocks or files is tracked for changes between snapshots. This may be done by a file system filter running in kernel mode. The data blocks or files that are tagged as unchanged are not transferred to backup because there is no need to update since the last backup. Other data blocks and files may be first tested for change, for example by comparing digital fingerprints of the current data versus the previously backed up data, before transferring to backup.
1 . A method for data management, comprising:
transmitting, from a data management system (DMS) to a compute infrastructure, signaling that instructs the compute infrastructure to take a snapshot of a machine of the compute infrastructure;
determining a subset of data blocks of the machine to transfer to the DMS for backup of the snapshot based at least in part on tracking data generated during a tracking session, wherein the tracking data indicates that the subset of data blocks were write accessed during the tracking session, and wherein the tracking session begins based at least in part on a snapshot of the machine previous to the snapshot and ends in response to the compute infrastructure taking the snapshot; and
causing the transfer of the subset of data blocks to the DMS from the compute infrastructure based at least in part on the determination.
2 . The method of claim 1 , wherein determining the subset of data blocks to transfer to the DMS comprises:
determining that data blocks of the subset of data blocks were write accessed during the tracking session based at least in part on the tracking data, wherein the subset of data blocks are transferred based at least in part on being write accessed during the tracking session.
3 . The method of claim 1 , wherein determining the subset of data blocks to transfer to the DMS comprises:
determining that data blocks of the subset of data blocks were write accessed during the tracking session based at least in part on the tracking data;
causing a transfer of respective first digital fingerprints of the data blocks associated with the previous snapshot to the compute infrastructure; and
instructing the compute infrastructure to determine whether the respective first digital fingerprints and respective current digital fingerprints of the data blocks are the same,
wherein the subset of data blocks are transferred from the compute infrastructure to the DMS based at least in part on the respective first digital fingerprints and respective current digital fingerprints being different.
4 . The method of claim 1 , further comprising:
determining a second subset of data blocks of the machine to refrain from transferring to the DMS as part of the backup of the snapshot based at least in part on the tracking data, wherein transferring the subset of data blocks to the DMS excludes the second subset of data blocks based at least in part on determining the second subset of data blocks.
5 . The method of claim 4 , wherein determining the second subset of data blocks comprises:
determining that the second subset of data blocks are tagged as unchanged based at least in part on the tracking data.
6 . The method of claim 4 , wherein determining the second subset of data blocks comprises:
determining that data blocks of the second subset of data blocks were write accessed during the tracking session based at least in part on the tracking data;
causing a transfer of respective first digital fingerprints of the data blocks associated with the previous snapshot to the compute infrastructure;
instructing the compute infrastructure to determine whether the respective first digital fingerprints and respective current digital fingerprints of the data blocks are the same; and
refraining from transferring the second subset of data blocks from the compute infrastructure to the DMS based at least in part on the respective first digital fingerprints and respective current digital fingerprints being the same.
7 . The method of claim 1 , further comprising:
receiving, at the DMS, the tracking data from a file system filter of the compute infrastructure in response to compute infrastructure taking the snapshot.
8 . The method of claim 1 , wherein:
the DMS comprises a DMS agent at the compute infrastructure and a DMS cluster, and
the DMS agent coordinates the transfer of the subset of data blocks to the DMS cluster based at least in part on the tracking data.
9 . The method of claim 8 , further comprising:
instructing the DMS agent to begin the tracking session based at least in part on the previous snapshot and to end the tracking session in response to the compute infrastructure taking the snapshot.
10 . The method of claim 8 , further comprising:
instructing the DMS agent to begin a second tracking session based at least in part on the compute infrastructure taking the snapshot.
11 . The method of claim 8 , wherein the DMS agent receives the tracking data and determines the subset of data blocks to transfer to the DMS cluster based at least in part on the tracking data.
12 . An apparatus for data management, comprising:
at least one processor;
memory coupled with the at least one processor; and
instructions stored in the memory and executable by the at least one processor to cause the apparatus to:
transmit, from a data management system (DMS) to a compute infrastructure, signaling that instructs the compute infrastructure to take a snapshot of a machine of the compute infrastructure;
determine a subset of data blocks of the machine to transfer to the DMS for backup of the snapshot based at least in part on tracking data generated during a tracking session, wherein the tracking data indicates that the subset of data blocks were write accessed during the tracking session, and wherein the tracking session based at least in part on a snapshot of the machine previous to the snapshot and ends in response to the compute infrastructure taking the snapshot; and
cause the transfer of the subset of data blocks to the DMS from the compute infrastructure based at least in part on the determination.
13 . The apparatus of claim 12 , wherein the instructions to determine the subset of data blocks to transfer to the DMS are executable by the at least one processor to cause the apparatus to:
determine that data blocks of the subset of data blocks were write accessed during the tracking session based at least in part on the tracking data, wherein the subset of data blocks are transferred based at least in part on being write accessed during the tracking session.
14 . The apparatus of claim 12 , wherein the instructions to determine the subset of data blocks to transfer to the DMS are executable by the at least one processor to cause the apparatus to:
determine that data blocks of the subset of data blocks were write accessed during the tracking session based at least in part on the tracking data;
cause a transfer of respective first digital fingerprints of the data blocks associated with the previous snapshot to the compute infrastructure; and
instruct the compute infrastructure to determine whether the respective first digital fingerprints and respective current digital fingerprints of the data blocks are the same,
wherein the subset of data blocks are transferred from the compute infrastructure to the DMS based at least in part on the respective first digital fingerprints and respective current digital fingerprints being different.
15 . The apparatus of claim 12 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
determine a second subset of data blocks of the machine to refrain from transferring to the DMS as part of the backup of the snapshot based at least in part on the tracking data, wherein transferring the subset of data blocks to the DMS excludes the second subset of data blocks based at least in part on determining the second subset of data blocks.
16 . The apparatus of claim 15 , wherein the instructions to determine the second subset of data blocks are executable by the at least one processor to cause the apparatus to:
determine that the second subset of data blocks are tagged as unchanged based at least in part on the tracking data.
17 . The apparatus of claim 15 , wherein the instructions to determine the second subset of data blocks are executable by the at least one processor to cause the apparatus to:
determine that data blocks of the second subset of data blocks were write accessed during the tracking session based at least in part on the tracking data;
cause a transfer of respective first digital fingerprints of the data blocks associated with the previous snapshot to the compute infrastructure;
instruct the compute infrastructure to determine whether the respective first digital fingerprints and respective current digital fingerprints of the data blocks are the same; and
refrain from transferring the second subset of data blocks from the compute infrastructure to the DMS based at least in part on the respective first digital fingerprints and respective current digital fingerprints being the same.
18 . The apparatus of claim 12 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
receive, at the DMS, the tracking data from a file system filter of the compute infrastructure in response to compute infrastructure taking the snapshot.
19 . The apparatus of claim 12 , wherein:
the DMS comprises a DMS agent at the compute infrastructure and a DMS cluster, and
the DMS agent coordinates the transfer of the subset of data blocks to the DMS cluster based at least in part on the tracking data.
20 . A non-transitory computer-readable medium storing code for data management, the code comprising instructions executable by at least one processor to:
transmit, from a data management system (DMS) to a compute infrastructure, signaling that instructs the compute infrastructure to take a snapshot of a machine of the compute infrastructure;
determine a subset of data blocks of the machine to transfer to the DMS for backup of the snapshot based at least in part on tracking data generated during a tracking session, wherein the tracking data indicates that the subset of data blocks were write accessed during the tracking session, and wherein the tracking session based at least in part on a snapshot of the machine previous to the snapshot and ends in response to the compute infrastructure taking the snapshot; and
cause the transfer of the subset of data blocks to the DMS from the compute infrastructure based at least in part on the determination.