Accessing file system replica during ongoing replication operations
A technique for accessing replicas of file systems generates a snap (point-in-time version) of a replica on a destination site and provides access to the snap for reading and/or writing while replication operations on the replica between a source site and the destination site continue to be performed. As the snap is a separate object from the replica itself, users may access the snap without affecting the replica or disturbing ongoing replication activities. Users can also mount the snap to perform destination-driven backup operations, to test disaster recovery, and/or to perform other activities.
1. A method of accessing file system replicas, the method comprising:
conducting replication operations on a file system hosted on a first data storage system repeatedly to update a replica of the file system on a second data storage system to reflect changes in the file system that occur in response to activities at the first data storage system over time;
in a course of conducting the replication operations, generating, by the second data storage system, a snap of the replica, the snap providing a point-in-time version of the replica of the file system; and
while the replication operations continue to update the replica, directing IO requests received by the second data storage system to the snap of the replica to effect read and/or write operations on the snap of the replica;
wherein conducting the replication operations includes performing discrete update operations on the replica to keep the replica current with changes made to the file system, wherein the first data storage system has a transaction log for storing pending metadata transactions for the file system, and wherein the method further comprises, prior to the second data storage system generating the snap:
the first data storage system applying the pending metadata transactions from the transaction log to the file system; and
after applying the pending metadata changes to the file system, the first data storage system sending a marker to the second data storage system, the marker identifying a point relative to the discrete update operations at which the pending metadata changes have been applied to the file system,
wherein generating the snap by the second data storage system is performed only after the second data storage system has received the marker.
2. The method of claim 1 , wherein the method further comprises:
realizing the file system in a form of a container file stored in an internal file system in the first data storage system,
realizing the replica in a form of a first container file stored in an internal file system in the second data storage system; and
realizing the snap of the replica in a form of a second container file stored in the internal file system in the second data storage system.
3. The method of claim 2 , wherein the method further comprises, prior to directing the IO requests to the snap, the second data storage system applying the pending metadata transactions from the transaction log to the snap to generate a consistent image of the file system in the snap.
4. The method of claim 3 , further comprising realizing the transaction log as an embedded transaction log embedded within the file system,
wherein the replica and the snap on the second data storage system each include a version of the embedded transaction log, and wherein applying the pending transactions from the transaction log to the snap includes reading the pending transactions from embedded transaction log in the snap and applying the pending transactions to the point-in-time version of the replica of the file system in the snap to generate the consistent image of the file system in the snap.
5. The method of claim 3 , further comprising:
realizing the transaction log in a second container file in the internal file system in the first data storage system; and
forming a consistency group that includes both the first container file and the second container file in the internal file system in the first data storage system,
wherein conducting the replication operations includes performing operations on the consistency group to generate both the replica of the file system and a replica of the transaction log, and
wherein applying the pending transactions from the transaction log to the snap includes reading the pending transactions from the replica of the transaction log and applying the pending transactions to the point-in-time version of the replica of the file system to generate the consistent image of the file system in the snap.
6. The method of claim 2 , wherein the method further comprises generating, by the second data storage system, additional snaps of the replica, each additional snap providing a respective point-in-time version of the replica of the file system at a respective point in time.
7. The method of claim 6 , further comprising, after generating each additional snap of the file system, (i) applying all pending metadata transactions from the transaction log to the additional snap and (ii) deleting the immediately preceding snap.
8. The method of claim 6 , further comprising the second data storage system directing a backup copy to be made of any of the snap and the additional snaps.
9. The method of claim 2 , wherein directing the IO requests to the snap includes directing both read and write IO requests to the snap to effect read and write operations on the snap, respectively, wherein at least some of the IO requests are part of a test of the replica to assess whether the replica can provide hosts with access to the file system in an event that the first data storage system is unable to provide access to the file system.
10. The method of claim 2 , wherein the replication operations include continuous replication operations that mirror respective IO requests directed to the file system to the replica, wherein the IO requests are grouped into bunches, each bunch including multiple consecutive IO requests, and wherein generating the snap is performed only after applying all IO requests in a bunch to the replica and before applying any IO requests in any subsequent bunch to the replica.
11. A system comprising a first data storage system and a second data storage system coupled to the first data storage system, the first data storage system and the second data storage system each including a respective set of processors and memory, the system constructed and arranged to:
conduct replication operations on a file system hosted on the first data storage system repeatedly to update a replica of the file system on the second data storage system to reflect changes in the file system that occur in response to activities at the first data storage system over time;
in a course of conducting the replication operations, generate, by the second data storage system, a snap of the replica, the snap providing a point-in-time version of the replica of the file system; and
while the replication operations continue to update the replica, direct 10 requests received by the second data storage system to the snap of the replica to effect read and/or write operations on the snap of the replica,
wherein the first data storage system has a transaction log for storing pending metadata transactions for the file system, wherein, when constructed and arranged to conduct the replication operations, system is further constructed and arranged to perform discrete update operations on the replica to keep the replica current with changes made to the file system, and wherein the system is further constructed and arranged to, prior to the second data storage system generating the snap:
apply, by the first data storage system, the pending metadata transactions from the transaction log to the file system; and
after applying the pending metadata changes to the file system, send, by the first data storage system, a marker to the second data storage system, the marker identifying a point relative to the discrete update operations at which the pending metadata changes have been applied to the file system,
wherein the system is constructed and arranged to generate the snap by the second data storage system only after the second data storage system has received the marker.
12. The system of claim 11 , wherein the system is further constructed and arranged to, prior to directing the IO requests to the snap, apply the pending metadata transactions from the transaction log to the snap to generate a consistent image of the file system in the snap.
13. A computer program product including a set of non-transitory, computer-readable media having instructions which, when executed by one or more processors, cause the processors to perform a method of accessing file system replicas, the method comprising:
conducting replication operations on a file system hosted on a first data storage system repeatedly to update a replica of the file system on a second data storage system to reflect changes in the file system that occur in response to activities at the first data storage system over time;
in a course of conducting the replication operations, generating, by the second data storage system, a snap of the replica, the snap providing a point-in-time version of the replica of the file system; and
while the replication operations continue to update the replica, directing 10 requests received by the second data storage system to the snap of the replica to effect read and/or write operations on the snap of the replica,
wherein the first data storage system has a transaction log for storing pending metadata transactions for the file system,
wherein conducting the replication operations includes performing discrete update operations on the replica to keep the replica current with changes made to the file system, and wherein the method further comprises, prior to the second data storage system generating the snap:
the first data storage system applying the pending metadata transactions from the transaction log to the file system; and
after applying the pending metadata changes to the file system, the first data storage system sending a marker to the second data storage system, the marker identifying a point relative to the discrete update operations at which the pending metadata changes have been applied to the file system,
wherein generating the snap by the second data storage system is performed only after the second data storage system has received the marker.
14. The computer program product of claim 13 , wherein the method further comprises, prior to directing the IO requests to the snap, applying the pending metadata transactions from the transaction log to the snap to generate a consistent image of the file system in the snap.
15. The method of claim 1 , wherein the first data storage system is coupled to the second data storage system over a network, and wherein the first data storage system and the second data storage system each include a respective set of processing units and memory.