Restoration of snapshots from cloud using differential snapshot mechanism
A method comprises receiving a request to restore a given snapshot of a plurality of snapshots from a cloud storage platform to a storage array. In the method, a first metadata file corresponding to the given snapshot is downloaded from the cloud storage platform, and data corresponding to the given snapshot is written to the storage array. An additional metadata file corresponding to a previous snapshot of the plurality of snapshots with respect to the given snapshot is downloaded. The first metadata file and the additional metadata file are compared to determine whether one or more offsets in the additional metadata file are present in the first metadata file. Data corresponding to any of the one or more offsets determined to be absent from the first metadata file is written to the storage array, and the one or more absent offsets are added to the first metadata file.
1. An apparatus comprising:
at least one processing platform comprising a plurality of processing devices;
said at least one processing platform being configured:
to receive a request to restore a given snapshot of a plurality of snapshots from a cloud storage platform to a storage array;
to download a first metadata file corresponding to the given snapshot from the cloud storage platform;
to write data corresponding to the given snapshot to the storage array;
to download an additional metadata file corresponding to a previous snapshot of the plurality of snapshots with respect to the given snapshot;
to compare the first metadata file and the additional metadata file to determine whether one or more offsets in the additional metadata file are present in the first metadata file;
to write to the storage array data corresponding to any of the one or more offsets determined to be absent from the first metadata file; and
to add to the first metadata file the one or more offsets determined to be absent from the first metadata file;
wherein said at least one processing platform is further configured to send the plurality of snapshots to the cloud storage platform for storage on the cloud storage platform; and
wherein, in sending the plurality of snapshots to the cloud storage platform for storage, said at least one processing platform is configured:
to read data corresponding to respective ones of the plurality of snapshots from the storage array; and
to compress the data corresponding to the respective ones of the plurality of snapshots using a minimum data block size.
2. The apparatus of claim 1 wherein said at least one processing platform is further configured:
to recursively download respective additional metadata files corresponding to respective previous snapshots of the plurality of snapshots with respect to the given snapshot and to the previous snapshot; and
to recursively compare the first metadata file and the respective additional metadata files to determine whether one or more offsets in the respective additional metadata files are present in the first metadata file.
3. The apparatus of claim 2 wherein said at least one processing platform is further configured:
to recursively write to the storage array data corresponding to any of the one or more offsets in the respective additional metadata files determined to be absent from the first metadata file; and
to recursively add to the first metadata file the one or more offsets in the respective additional metadata files determined to be absent from the first metadata file.
4. The apparatus of claim 3 wherein said at least one processing platform is further configured to recursively perform the downloading of the respective additional metadata files, the comparing of the first metadata file and the respective additional metadata files, the writing of the data corresponding to the one or more offsets in the respective additional metadata files and the adding of the one or more offsets in the respective additional metadata files until a base snapshot is reached.
5. The apparatus of claim 1 wherein said at least one processing platform is further configured to ignore any of the one or more offsets determined to be present in the first metadata file.
6. The apparatus of claim 5 wherein said at least one processing platform is further configured to prevent writing to the storage array data corresponding to the one or more offsets determined to be present in the first metadata file.
7. The apparatus of claim 1 wherein, in sending the plurality of snapshots to the cloud storage platform for storage, said at least one processing platform is further configured to encrypt the data corresponding to the respective ones of the plurality of snapshots using the minimum data block size.
8. The apparatus of claim 1 , wherein the minimum data block size is about 8 kilobytes.
9. The apparatus of claim 1 wherein, in sending the plurality of snapshots to the cloud storage platform for storage, said at least one processing platform is further configured:
to store the compressed data corresponding to respective ones of the plurality of snapshots as a plurality of data files in a cache using the minimum data block size; and
to send the plurality of data files to the cloud storage platform once a cache storage threshold is reached.
10. The apparatus of claim 9 wherein, in sending the plurality of snapshots to the cloud storage platform for storage, said at least one processing platform is further configured:
to generate respective metadata files for the respective ones of the plurality of snapshots;
wherein the respective metadata files comprise information about array and cloud offsets for the respective ones of the plurality of snapshots.
11. The apparatus of claim 10 wherein, in sending the plurality of snapshots to the cloud storage platform for storage, said at least one processing platform is further configured to send the respective metadata files to the cloud storage platform once the cache storage threshold is reached.
12. The apparatus of claim 1 wherein said at least one processing platform comprises a cloud tiering appliance.
13. A method comprising:
receiving a request to restore a given snapshot of a plurality of snapshots from a cloud storage platform to a storage array;
downloading a first metadata file corresponding to the given snapshot from the cloud storage platform;
writing data corresponding to the given snapshot to the storage array;
downloading an additional metadata file corresponding to a previous snapshot of the plurality of snapshots with respect to the given snapshot;
comparing the first metadata file and the additional metadata file to determine whether one or more offsets in the additional metadata file are present in the first metadata file;
writing to the storage array data corresponding to any of the one or more offsets determined to be absent from the first metadata file; and
adding to the first metadata file the one or more offsets determined to be absent from the first metadata file;
the method further comprising:
sending the plurality of snapshots to the cloud storage platform for storage on the cloud storage platform;
wherein sending the plurality of snapshots to the cloud storage platform for storage comprises:
reading data corresponding to respective ones of the plurality of snapshots from the storage array; and
compressing the data corresponding to the respective ones of the plurality of snapshots using a minimum data block size;
wherein the method is performed by at least one processing platform comprising at least one processing device comprising a processor coupled to a memory.
14. The method of claim 13 further comprising:
recursively downloading respective additional metadata files corresponding to respective previous snapshots of the plurality of snapshots with respect to the given snapshot and to the previous snapshot; and
recursively comparing the first metadata file and the respective additional metadata files to determine whether one or more offsets in the respective additional metadata files are present in the first metadata file.
15. The method of claim 14 further comprising:
recursively writing to the storage array data corresponding to any of the one or more offsets in the respective additional metadata files determined to be absent from the first metadata file; and
recursively adding to the first metadata file the one or more offsets in the respective additional metadata files determined to be absent from the first metadata file.
16. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing platform causes said at least one processing platform:
to receive a request to restore a given snapshot of a plurality of snapshots from a cloud storage platform to a storage array;
to download a first metadata file corresponding to the given snapshot from the cloud storage platform;
to write data corresponding to the given snapshot to the storage array;
to download an additional metadata file corresponding to a previous snapshot of the plurality of snapshots with respect to the given snapshot;
to compare the first metadata file and the additional metadata file to determine whether one or more offsets in the additional metadata file are present in the first metadata file;
to write to the storage array data corresponding to any of the one or more offsets determined to be absent from the first metadata file; and
to add to the first metadata file the one or more offsets determined to be absent from the first metadata file;
wherein the program code when executed by said at least one processing platform further causes said at least one processing platform to send the plurality of snapshots to the cloud storage platform for storage on the cloud storage platform; and
wherein, in sending the plurality of snapshots to the cloud storage platform for storage, said at least one processing platform is configured:
to read data corresponding to respective ones of the plurality of snapshots from the storage array; and
to compress the data corresponding to the respective ones of the plurality of snapshots using a minimum data block size.
17. The computer program product according to claim 16 wherein the program code further causes said at least one processing platform:
to recursively download respective additional metadata files corresponding to respective previous snapshots of the plurality of snapshots with respect to the given snapshot and to the previous snapshot; and
to recursively compare the first metadata file and the respective additional metadata files to determine whether one or more offsets in the respective additional metadata files are present in the first metadata file.
18. The computer program product according to claim 17 wherein the program code further causes said at least one processing platform:
to recursively write to the storage array data corresponding to any of the one or more offsets in the respective additional metadata files determined to be absent from the first metadata file; and
to recursively add to the first metadata file the one or more offsets in the respective additional metadata files determined to be absent from the first metadata file.
19. The computer program product according to claim 16 wherein, in sending the plurality of snapshots to the cloud storage platform for storage, said at least one processing platform is further configured to encrypt the data corresponding to the respective ones of the plurality of snapshots using the minimum data block size.
20. The computer program product according to claim 16 wherein, in sending the plurality of snapshots to the cloud storage platform for storage, said at least one processing platform is further configured:
to store the compressed data corresponding to respective ones of the plurality of snapshots as a plurality of data files in a cache using the minimum data block size; and
to send the plurality of data files to the cloud storage platform once a cache storage threshold is reached.