Granular replication of volume subsets
Data is replicated on a backup node, where the granularity of the replication can be less than a full volume. A data consistency group comprising a subset of data for a volume is defined for a primary node. A set of differences for the data consistency group is sent to a backup node. The backup node creates change logs in response to receiving the set of differences. In response to receiving a request to access a file having data in the data consistency group, the backup node creates a clone of the file. The backup node determines whether an update to a data block of the file exists in the change logs. In response to determining that the update to the data block exists in the change logs, the backup node updates a copy of the data block for the cloned file with data in the change logs.
1. A method, comprising:
evaluating a first snapshot of a volume at a first point in time and a second snapshot of the volume at a second point in time to identify differences of the volume between the first point in time and the second point in time, wherein the differences are identified at a volume level for data of the volume;
utilizing configuration data, defining a subset of the data of the volume as a consistency group that includes less than all of the data of the volume, to filter the differences to generate a set of differences for the consistency group within the volume, wherein the filtering removes differences identified for data of the volume not included within the consistency group; and
replicating the set of differences, excluding the differences identified for data of the volume not included within the consistency group, to a destination node for creating a change log having a domain at a consistency group level or sub-consistency group level.
2. The method of claim 1 , comprising:
filtering the differences at a block level granularity to exclude blocks that are the same between the first snapshot and the second snapshot.
3. The method of claim 1 , comprising:
utilizing the configuration data during snapshot creation so that the first snapshot includes data of the consistency group and exclude other data within the volume that is not part of the consistency group.
4. The method of claim 1 , comprising:
utilizing the configuration data during snapshot creation so that the second snapshot includes data of the consistency group and exclude other data within the volume that is not part of the consistency group.
5. The method of claim 1 , comprising:
transmitting the set of differences of over a network to the destination node while preserving block sharing amongst blocks storing data of the volume.
6. The method of claim 1 , comprising:
transmitting the set of differences of over a network to the destination node while retaining compression of blocks storing data of the volume.
7. The method of claim 1 , comprising:
defining the change log to have the domain set for a file system within the consistency group.
8. The method of claim 1 , comprising:
defining the change log to have the domain set for a logical unit number (LUN) within the consistency group.
9. The method of claim 1 , comprising:
defining the change log to have the domain set for a virtual machine disk within the consistency group.
10. A non-transitory machine readable medium having stored thereon instructions comprising machine executable code which when executed by a machine, cause the machine to:
evaluate a first snapshot of a volume at a first point in time and a second snapshot of the volume at a second point in time to identify differences of the volume between the first point in time and the second point in time, wherein the differences are identified at a volume level for data of the volume;
utilize configuration data, defining a subset of the data of the volume as a consistency group that includes less than all of the data of the volume, to filter the differences to generate a set of differences for the consistency group within the volume, wherein the filtering removes differences identified for data of the volume not included within the consistency group; and
replicate the set of differences, excluding the differences identified for data of the volume not included within the consistency group, to a destination node for creating a change log having a domain at a consistency group level or sub-consistency group level.
11. The non-transitory machine readable medium of claim 10 , wherein the machine executable code causes the machine to:
apply data operations to change data at the destination node in relation to the set of differences; and
log metadata blocks in the change log specify information about the set of differences.
12. The non-transitory machine readable medium of claim 10 , wherein the machine executable code causes the machine to:
determine whether a condition exists to trigger the application of the change log to data of the consistency group maintained at the destination node; and
in response to detecting the condition, apply the change log to the data of the consistency group maintained at the destination node.
13. The non-transitory machine readable medium of claim 10 , wherein the machine executable code causes the machine to:
utilize the configuration data during snapshot creation so that the first snapshot includes data of the consistency group and exclude other data within the volume that is not part of the consistency group.
14. An apparatus comprising:
a processor; and
a machine readable storage medium having machine executable code stored therein that is executable by the processor to cause the apparatus to:
evaluate a first snapshot of a volume at a first point in time and a second snapshot of the volume at a second point in time to identify differences of the volume between the first point in time and the second point in time, wherein the differences are identified at a volume level for data of the volume;
utilize configuration data, defining a subset of the data of the volume as a consistency group that includes less than all of the data of the volume, to filter the differences to generate a set of differences for the consistency group within the volume, wherein the filtering removes differences identified for data of the volume not included within the consistency group; and
replicate the set of differences, excluding the differences identified for data of the volume not included within the consistency group, to a destination node for creating a change log having a domain at a consistency group level or sub-consistency group level.
15. The apparatus of claim 14 , wherein the machine executable code causes the apparatus to:
filter the differences at a block level granularity to exclude blocks that are the same between the first snapshot and the second snapshot.
16. The apparatus of claim 14 , wherein the machine executable code causes the apparatus to:
utilize the configuration data during snapshot creation so that the first snapshot includes data of the consistency group and exclude other data within the volume that is not part of the consistency group.
17. The apparatus of claim 14 , wherein the machine executable code causes the apparatus to:
utilize the configuration data during snapshot creation so that the second snapshot includes data of the consistency group and exclude other data within the volume that is not part of the consistency group.
18. The apparatus of claim 14 , wherein the machine executable code causes the apparatus to:
transmit the set of differences over a network to the destination node while preserving block sharing amongst blocks storing data of the volume.
19. The apparatus of claim 14 , wherein the machine executable code causes the apparatus to:
transmit the set of differences over a network to the destination node while retaining compression of blocks storing data of the volume.
20. The apparatus of claim 14 , wherein the machine executable code causes the apparatus to:
define the change log to have the domain set for a file system within the consistency group.