Archiving NAS servers to the cloud
A technique for archiving NAS (network attached storage) servers includes replicating multiple locally-backed volumes, which support respective file systems of a NAS server, to respective cloud-backed volumes backed by a cloud-based data store. After replication has updated the cloud-backed volumes with contents from the locally-backed volumes, the technique further includes performing a group snapshot operation on the cloud-backed volumes. The group snapshot operation creates a point-in-time version of the cloud-backed volumes, which provides a replica of the NAS server archived in the cloud.
1. A method of archiving NAS (network attached storage) servers, the method comprising:
receiving a request to archive a NAS server in a data storage system, the NAS server including a set of multiple file systems, each of the set of file systems of the NAS server deployed on a respective, locally-backed volume in the data storage system;
in response to receiving the request, establishing a respective replication session on each of the locally-backed volumes, each replication session designating (i) a replication source as a respective one of the locally-backed volumes and (ii) a replication target as a respective cloud-backed volume, the cloud-backed volume backed by storage in a cloud-based data store; and
after the replication sessions have updated the cloud-backed volumes with contents from the locally-backed volumes on which the file systems of the NAS server are deployed, performing a group snapshot operation, the group snapshot operation generating, at a particular point in time, a snapshot of each of the cloud-backed volumes, each snapshot providing a new volume backed by the cloud-based data store, the snapshots generated by the group snapshot operation together providing an archived, point-in-time version of the NAS server,
wherein each of the replication sessions is a snapshot-shipping replication session configured to update a respective cloud-backed volume with contents of a respective locally-backed volume on a periodic basis.
2. The method of claim 1 , further comprising, after updating the cloud-backed volumes with new contents from the locally-backed volumes, performing a second group snapshot operation, the second group snapshot operation generating, at a later point in time, a second snapshot of each of the cloud-backed volumes, the second snapshots generated by the second group snapshot operation together providing a second archived, point-in-time version of the NAS server.
3. The method of claim 2 , further comprising storing a searchable metadata element for each snapshot, each searchable metadata element associated with a respective snapshot and identifying a NAS server and version, wherein different versions indicate different group snapshot operations.
4. The method of claim 3 , wherein each of the searchable metadata elements includes a NAS server identifier and a version number, and wherein the searchable metadata elements for all snapshots taken for a NAS server during a single group snapshot operation identify the same NAS server identifier and the same version number.
5. The method of claim 4 , wherein each searchable metadata element for a snapshot further includes:
a timestamp that indicates a time when the snapshot was generated; and
a name of the file system deployed upon the locally-backed volume that was replicated to the cloud-backed volume from which the snapshot was generated.
6. The method of claim 4 , further comprising identifying a particular archived version of the NAS server by querying the searchable metadata elements based on NAS server identifier and version number.
7. The method of claim 3 , wherein the data store organizes data in objects, and wherein the method further comprises storing the data of each snapshot generated during the first group snapshot operation as a respective set of objects in the data store.
8. The method of claim 7 , further comprising maintaining data-sharing relationships in the data store, at least one of the cloud-backed volumes thus sharing data storage with a snapshot of that cloud-backed volume.
9. The method of claim 3 , wherein performing an update to a cloud-backed volume based on changes to a locally-backed volume includes:
taking a snap of the locally-backed volume acting as the replication source;
calculating a difference map that identifies differences between the snap and a previous snap of the same locally-backed volume taken earlier;
shipping the difference map to the replication destination, and
updating the cloud-backed volume to include changes identified in the difference map.
10. The method of claim 9 , wherein the replication sessions are managed by a replication manager in the data storage system, and wherein the group snapshot operation and the second group snapshot operation are performed in response to requests from the replication manager.
11. The method of claim 10 , further comprising the replication manager timing the requests to perform the group snapshot operations to occur after one set of snapshot shipping updates to the cloud-backed volumes finishes but prior to another set of snapshot shipping updates to the cloud-backed volumes beginning.
12. A computerized apparatus, comprising control circuitry constructed and arranged to:
receive a request to archive a NAS server in a data storage system, the NAS server including a set of multiple file systems, each of the set of file systems of the NAS server deployed on a respective, locally-backed volume in the data storage system;
in response to receiving the request, establish a respective replication session on each of the locally-backed volumes, each replication session designating (i) a replication source as a respective one of the locally-backed volumes and (ii) a replication target as a respective cloud-backed volume, the cloud-backed volume backed by storage in a cloud-based data store; and
after the replication sessions have updated the cloud-backed volumes with contents from the locally-backed volumes on which the file systems of the NAS server are deployed, perform a group snapshot operation, the group snapshot operation generating, at a particular point in time, a snapshot of each of the cloud-backed volumes, each snapshot providing a new volume backed by the cloud-based data store, the snapshots generated by the group snapshot operation together providing an archived, point-in-time version of the NAS server,
wherein each of the replication sessions is a snapshot-shipping replication session configured to update a respective cloud-backed volume with contents of a respective locally-backed volume on a periodic basis.
13. A computer program product including a set of non-transitory, computer-readable media having instructions which, when executed by control circuitry of a computerized apparatus, cause the computerized apparatus to perform a method of archiving NAS (network attached storage) servers, the method comprising:
receiving a request to archive a NAS server in a data storage system, the NAS server including a set of multiple file systems, each of the set of file systems of the NAS server deployed on a respective, locally-backed volume in the data storage system;
in response to receiving the request, establishing a respective replication session on each of the locally-backed volumes, each replication session designating (i) a replication source as a respective one of the locally-backed volumes and (ii) a replication target as a respective cloud-backed volume, the cloud-backed volume backed by storage in a cloud-based data store; and
after the replication sessions have updated the cloud-backed volumes with contents from the locally-backed volumes on which the file systems of the NAS server are deployed, performing a group snapshot operation, the group snapshot operation generating, at a particular point in time, a snapshot of each of the cloud-backed volumes, each snapshot providing a new volume backed by the cloud-based data store, the snapshots generated by the group snapshot operation together providing an archived, point-in-time version of the NAS server,
wherein each of the replication sessions is a snapshot-shipping replication session configured to update a respective cloud-backed volume with contents of a respective locally-backed volume on a periodic basis.
14. The computer program product of claim 13 , wherein the method further comprises, after updating the cloud-backed volumes with new contents from the locally-backed volumes, performing a second group snapshot operation, the second group snapshot operation generating, at a later point in time, a second snapshot of each of the cloud-backed volumes, the second snapshots generated by the second group snapshot operation together providing a second archived, point-in-time version of the NAS server.
15. The computer program product of claim 14 , wherein the method further comprises storing a searchable metadata element for each snapshot, each searchable metadata element associated with a respective snapshot and identifying a NAS server and version, wherein different versions indicate different group snapshot operations.
16. The computer program product of claim 15 , wherein the data store organizes data in objects, and wherein the method further comprises storing the data of each snapshot generated during the first group snapshot operation as a respective set of objects in the data store.
17. The computer program product of claim 16 , wherein the method further comprises maintaining data-sharing relationships in the data store, at least one of the cloud-backed volumes thus sharing data storage with a snapshot of that cloud-backed volume.
18. The computer program product of claim 15 , wherein performing an update to a cloud-backed volume based on changes to a locally-backed volume includes:
taking a snap of the locally-backed volume acting as the replication source;
calculating a difference map that identifies differences between the snap and a previous snap of the same locally-backed volume taken earlier;
shipping the difference map to the replication destination, and
updating the cloud-backed volume to include changes identified in the difference map.
19. The computer program product of claim 18 , wherein the replication sessions are managed by a replication manager in the data storage system, and wherein the group snapshot operation and the second group snapshot operation are performed in response to requests from the replication manager.
20. The computer program product of claim 19 , wherein the method further comprises the replication manager timing the requests to perform the group snapshot operations to occur after one set of snapshot shipping updates to the cloud-backed volumes finishes but prior to another set of snapshot shipping updates to the cloud-backed volumes beginning.