Data storage system with LUN snapshot shipping using volume-to-object translation

1. A method of shipping local snapshots of logical units of data storage (LUNs) to cloud storage, the local snapshots residing on local physical storage of a data storage system, comprising:

replicating a baseline local snapshot of a LUN to a cloud-backed LUN backed by cloud physical storage of a separate cloud storage system, and upon completion of the replicating, taking a first snapshot of the cloud-backed LUN to establish a baseline cloud-backed snapshot backed by the cloud physical storage; and

for a subsequent local snapshot of the LUN, (1) identifying data content differences between the subsequent local snapshot and the baseline local snapshot, (2) updating the cloud-backed LUN by applying the data content differences thereto, and (3) taking a second snapshot of the cloud-backed LUN, the second snapshot being a cloud-backed copy of the subsequent local snapshot and containing, in the cloud physical storage, (a) first objects unique to the second snapshot and (b) second objects shared with the baseline cloud-backed snapshot,

wherein the data storage system includes a block provider providing host computer access to the LUNs and a cloud appliance providing access to the cloud storage system, the cloud appliance including a volume-to-object translator providing the cloud-backed LUN and cloud-backed snapshots using respective logical mappings to corresponding sets of objects,

and wherein the method further including maintaining connections between the block provider and the cloud appliance including a data transfer channel and a control channel, the data transfer channel employing a block storage protocol, the control channel being structured for more general-purpose communications including exchanging out-of-band requests and corresponding responses.

2. The method of claim 1 , wherein the cloud storage system supports storage of searchable metadata elements usable to associate cloud-backed LUNs with corresponding LUNs of the data storage system.

3. The method of claim 2 , wherein each searchable metadata element for a respective cloud-backed snapshot includes a LUN name, a LUN identifier, a version number, and a timestamp, the LUN name being a name of a LUN corresponding to the cloud-backed snapshot, the LUN identifier being an identifier of the LUN, the version number being a number incremented with each snapshot operation and indicating a version of the LUN, and the timestamp including a date and time when a snapshot operation producing the cloud-backed snapshot was performed.

4. The method of claim 1 , wherein the cloud appliance further includes a snapshot-shipping maintenance and orchestration unit (SSMO) and one or more cloud application program interfaces (APIs), the SSMO initiating and controlling the shipping of the local snapshots, the cloud APIs managing communications with the cloud storage system.

5. The method of claim 1 , wherein the data transfer channel is an iSCSI channel, and the control channel utilizes a representation state transfer (REST) interface.

6. The method of claim 1 , wherein the block provider and cloud appliance are included in respective virtual machines of a single server computer.

7. The method of claim 1 , wherein identifying the data content differences includes using a snap difference operation comparing data contents of the baseline local snapshot to the subsequent local snapshot to generate a difference map, the difference map identifying (1) all data blocks of the subsequent local snapshot that represent additions or modifications over contents of the baseline snapshot, and (2) all data blocks that represent deallocations or deletions from the baseline local snapshot.

8. The method of claim 1 , further including steps of a snapshot restore process by which a version of the local LUN is restored from the cloud storage system, the steps including:

making a cloud-backed snapshot available for copying to the data storage system the cloud-backed snapshot corresponding to the version of the local LUN being restored;

creating a new local LUN of the same size as the cloud-backed snapshot and making the new local LUN available for the copying;

copying data of the cloud-backed snapshot to the new local LUN; and

upon completion of the copying, making the new local LUN available for servicing host IOs.

9. The method of claim 1 , wherein the block provider and cloud appliance are loosely coupled via an external network.

10. The method of claim 1 , wherein the data storage system includes one or more additional nodes functioning as network attached storage (NAS) nodes, the NAS nodes being clients of the block provider and utilizing block storage provided thereby.

11. A data storage system having local physical storage, a computer-implemented block provider, and a computer-implemented cloud appliance, the block provider storing local snapshots of logical units of data storage (LUNs) on the local physical storage, the block provider and cloud appliance being co-operative to ship the local snapshots to cloud storage by:

replicating a baseline local snapshot of a LUN to a cloud-backed LUN backed by cloud physical storage of a separate cloud storage system, and upon completion of the replicating, taking a first snapshot of the cloud-backed LUN to establish a baseline cloud-backed snapshot backed by the cloud physical storage; and

for a subsequent local snapshot of the LUN, (1) identifying data content differences between the subsequent local snapshot and the baseline local snapshot, (2) updating the cloud-backed LUN by applying the data content differences thereto, and (3) taking a second snapshot of the cloud-backed LUN, the second snapshot being a cloud-backed copy of the subsequent local snapshot and containing, in the cloud physical storage, (a) first objects unique to the second snapshot and (b) second objects shared with the baseline cloud-backed snapshot,

wherein the cloud appliance includes a volume-to-object translator providing the cloud-backed LUN and cloud-backed snapshots using respective logical mappings to corresponding sets of objects,

and wherein the data storage system further includes a data transfer channel and a control channel between the block provider and the cloud appliance, the data transfer channel employing a block storage protocol, the control channel being structured for more general-purpose communications including exchanging out-of-band requests and corresponding responses.

12. The data storage system of claim 11 , wherein the cloud storage system supports storage of searchable metadata elements usable to associate cloud-backed LUNs with corresponding LUNs of the data storage system.

13. The data storage system of claim 12 , wherein each searchable metadata element for a respective cloud-backed snapshot includes a LUN name, a LUN identifier, a version number, and a timestamp, the LUN name being a name of a LUN corresponding to the cloud-backed snapshot, the LUN identifier being an identifier of the LUN, the version number being a number incremented with each snapshot operation and indicating a version number of the LUN, and the timestamp including a date and time when a snapshot operation producing the cloud-backed snapshot was performed.

14. The data storage system of claim 11 , wherein the cloud appliance further includes a snapshot-shipping maintenance and orchestration unit (SSMO) and one or more cloud application program interfaces (APIs), the SSMO initiating and controlling the shipping of the local snapshots, the cloud APIs managing communications with the cloud storage system.

15. The data storage system of claim 11 , wherein the data transfer channel is an iSCSI channel, and the control channel utilizes a representation state transfer (REST) interface.

16. The data storage system of claim 11 , wherein the block provider and cloud appliance are included in respective virtual machines of a single server computer.

17. The data storage system of claim 11 , wherein identifying the data content differences includes using a snap difference operation comparing data contents of the baseline local snapshot to the subsequent local snapshot to generate a difference map, the difference map identifying (1) all data blocks of the subsequent local snapshot that represent additions or modifications over contents of the baseline snapshot, and (2) all data blocks that represent deallocations or deletions from the baseline local snapshot.

18. The data storage system of claim 11 , wherein the block provider and cloud appliance are further co-operative to perform a snapshot restore process by which a version of the local LUN is restored from the cloud storage system, the snapshot restore process including:

making a cloud-backed snapshot available for copying to the data storage system the cloud-backed snapshot corresponding to the version of the local LUN being restored;

creating a new local LUN of the same size as the cloud-backed snapshot and making the new local LUN available for the copying;

copying data of the cloud-backed snapshot to the new local LUN; and

upon completion of the copying, making the new local LUN available for servicing host IOs.

19. The data storage system of claim 11 , wherein the block provider and cloud appliance are loosely coupled via an external network.

20. The data storage system of claim 11 , further including one or more additional nodes functioning as network attached storage (NAS) nodes, the NAS nodes being clients of the block provider and utilizing block storage provided thereby.