Cloud-based snapshots with coarse and fine granularity maps for changed data
View Patent ↗Data protection appliances are provided for the protection of at least one virtual machine. A plurality of snapshots of at least one virtual machine are obtained. Each snapshot comprises (i) production data of the virtual machine, and (ii) a fine granularity data structure for changed data that is maintained in volatile memory and records metadata for input/output (I/O) operations corresponding to changed data. A metadata differential is generated by aggregating, for example, the fine granularity data structures for any time intervals since the prior snapshot that were flushed to a storage volume and included in the prior snapshot. The storage volume is replicated using the metadata differential. The snapshots optionally further comprise a coarse granularity bit map for changed data indicating whether corresponding blocks of data have changed.
1. A method for protection of at least one virtual machine, comprising:
obtaining, by at least one processing device of a data protection appliance, a plurality of snapshots of said at least one virtual machine, wherein each of said plurality of snapshots comprise (i) production data of said at least one virtual machine, (ii) a fine granularity data structure for changed data, wherein said fine granularity data structure is maintained in volatile memory and records metadata for input/output (I/O) operations corresponding to said changed data, and (iii) a coarse granularity bit map stored for changed data, wherein the coarse granularity bit map comprises a plurality of bits each indicating whether corresponding blocks of data have changed, wherein the coarse granularity bit has a granularity that is more coarse than the granularity of the fine granularity data structure, and wherein said data protection appliance is external to a production network comprising said virtual machine;
generating a metadata differential, by said at least one processing device of said data protection appliance, by aggregating: (i) the fine granularity data structures for any time intervals since a prior snapshot that were flushed to at least one storage volume and included in said prior snapshot, and (ii) the coarse granularity bit maps since the prior snapshot; and
replicating said at least one storage volume using said metadata differential.
2. The method of claim 1 , wherein said coarse granularity bit map and said fine granularity data structure are generated by a Change Block Tracking driver executing on a guest operating system on said production network.
3. The method of claim 1 , wherein said data protection appliance obtains said plurality of snapshots for a plurality of said virtual machines.
4. The method of claim 1 , wherein said metadata comprises an offset and a length for each of said I/O operations.
5. The method of claim 1 , wherein said step of generating a metadata differential further comprises the step of obtaining, from a subsequent snapshot, said fine granularity data structures for said time intervals since said prior snapshot that were not flushed to said at least one storage volume and included in said prior snapshot.
6. The method of claim 1 , wherein said step of replicating said at least one storage volume using said metadata differential further comprises the step of using said metadata differential to read said changed data from a subsequent snapshot.
7. The method of claim 1 , wherein said step of obtaining said plurality of snapshots of said at least one virtual machine is performed using a snapshot mechanism provided by a cloud service where said at least one virtual machine executes.
8. The method of claim 1 , further comprising the step of flushing said fine granularity data structure after a predefined flush interval.
9. A system for protection of at least one virtual machine, comprising:
a memory; and
at least one processing device, coupled to the memory, operative to implement the following steps:
obtaining, by at least one processing device of a data protection appliance, a plurality of snapshots of said at least one virtual machine, wherein each of said plurality of snapshots comprise (i) production data of said at least one virtual machine, (ii) a fine granularity data structure for changed data, wherein said fine granularity data structure is maintained in volatile memory and records metadata for input/output (I/O) operations corresponding to said changed data, and (iii) a coarse granularity bit map stored for changed data, wherein the coarse granularity bit map comprises a plurality of bits each indicating whether corresponding blocks of data have changed, wherein the coarse granularity bit has a granularity that is more coarse than the granularity of the fine granularity data structure, and wherein said data protection appliance is external to a production network comprising said virtual machine;
generating a metadata differential, by said at least one processing device of said data protection appliance, by aggregating: (i) the fine granularity data structures for any time intervals since a prior snapshot that were flushed to at least one storage volume and included in said prior snapshot, and (ii) the coarse granularity bit maps since the prior snapshot; and
replicating said at least one storage volume using said metadata differential.
10. The system of claim 9 , wherein said coarse granularity bit map and said fine granularity data structure are generated by a Change Block Tracking driver executing on a guest operating system on said production network.
11. The system of claim 9 , wherein said data protection appliance obtains said plurality of snapshots for a plurality of said virtual machines.
12. The system of claim 9 , wherein said metadata comprises an offset and a length for each of said I/O operations.
13. The system of claim 9 , wherein said step of generating a metadata differential further comprises the step of obtaining, from a subsequent snapshot, said fine granularity data structures for said time intervals since said prior snapshot that were not flushed to said at least one storage volume and included in said prior snapshot.
14. The system of claim 9 , wherein said step of replicating said at least one storage volume using said metadata differential further comprises the step of using said metadata differential to read said changed data from a subsequent snapshot.
15. The system of claim 9 , wherein said step of obtaining said plurality of snapshots of said at least one virtual machine is performed using a snapshot mechanism provided by a cloud service where said at least one virtual machine executes.
16. The system of claim 9 , further comprising the step of flushing said fine granularity data structure after a predefined flush interval.
17. A computer program product for protection of at least one virtual machine, comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
obtaining, by at least one processing device of a data protection appliance, a plurality of snapshots of said at least one virtual machine, wherein each of said plurality of snapshots comprise (i) production data of said at least one virtual machine, (ii) a fine granularity data structure for changed data, wherein said fine granularity data structure is maintained in volatile memory and records metadata for input/output (I/O) operations corresponding to said changed data, and (iii) a coarse granularity bit map stored for changed data, wherein the coarse granularity bit map comprises a plurality of bits each indicating whether corresponding blocks of data have changed, wherein the coarse granularity bit has a granularity that is more coarse than the granularity of the fine granularity data structure, and wherein said data protection appliance is external to a production network comprising said virtual machine;
generating a metadata differential, by said at least one processing device of said data protection appliance, by aggregating: (i) the fine granularity data structures for any time intervals since a prior snapshot that were flushed to at least one storage volume and included in said prior snapshot, and (ii) the coarse granularity bit maps since the prior snapshot; and
replicating said at least one storage volume using said metadata differential.
18. The computer program product of claim 17 , wherein said coarse granularity bit map and said fine granularity data structure are generated by a Change Block Tracking driver executing on a guest operating system on said production network.
19. The computer program product of claim 17 , wherein said step of generating a metadata differential further comprises the step of obtaining, from a subsequent snapshot, said fine granularity data structures for said time intervals since said prior snapshot that were not flushed to said at least one storage volume and included in said prior snapshot.
20. The computer program product of claim 17 , wherein said step of replicating said at least one storage volume using said metadata differential further comprises the step of using said metadata differential to read said changed data from a subsequent snapshot.