File tiering with data integrity checks
A method comprises computing a first hash value for at least one file of a plurality of files to be tiered in connection with a file tiering operation of the plurality of files from a source storage location to a destination storage location, wherein the file tiering operation comprises tiering of the at least one file to the destination storage location. In the method, the first hash value is compared to a second hash value for the at least one file, wherein the second hash value is computed from data in an object on the destination storage location corresponding to the at least one file. Based at least in part on the comparing, a determination is made whether the first hash value is the same as the second hash value. The tiering of the at least one file to the destination storage location proceeds in response to an affirmative determination.
1 . An apparatus comprising:
at least one processing platform comprising at least one processing device;
said at least one processing platform being configured:
to compute, by executing one or more hashing algorithms, a first hash value for at least one file of a plurality of files to be tiered in connection with a file tiering operation of the plurality of files from a source storage location to a destination storage location, wherein computing the first hash value comprises computing a plurality of intermediate hash values for a plurality of data chunks of the at least one file, wherein the file tiering operation comprises writing the plurality of data chunks of the at least one file to an object on the destination storage location in response to computing the plurality of intermediate hash values;
to compare the first hash value to a second hash value for the at least one file, wherein the second hash value is computed from data in the object on the destination storage location corresponding to the at least one file;
to determine, based at least in part on the comparing, whether the first hash value is the same as the second hash value, wherein the first hash value and the second hash value are determined to be the same when the first hash value and the second hash value match; and
to proceed with tiering of the at least one file in response to an affirmative determination.
2 . The apparatus of claim 1 wherein said at least one processing platform is further configured to cause generation of a stub file for the at least one file on the source storage location in response to the affirmative determination.
3 . The apparatus of claim 2 wherein said at least one processing platform is further configured to cause deletion of the at least one file from the source storage location in response to the affirmative determination.
4 . The apparatus of claim 1 wherein said at least one processing platform is further configured to cause deletion of the object from the destination storage location corresponding to the at least one file in response to a negative determination.
5 . The apparatus of claim 1 wherein said at least one processing platform is further configured to generate a log entry indicating data corruption of the at least one file in response to a negative determination.
6 . The apparatus of claim 1 wherein, in response to a negative determination, said at least one processing platform is further configured to retry the tiering of the at least one file in at least one of another instance of the file tiering operation and another file tiering operation.
7 . The apparatus of claim 1 wherein said at least one processing platform is further configured to request the second hash value from a callback daemon.
8 . The apparatus of claim 7 wherein the request comprises:
a read request for the data in the object on the destination storage location; and
an object identifier indicating that the request corresponds to a data integrity check.
9 . The apparatus of claim 7 wherein the callback daemon reads the data in the object from the destination storage location and computes the second hash value.
10 . The apparatus of claim 1 wherein said at least one processing platform is further configured: to store the first hash value in a database; and to delete the first hash value from the database following the determining of whether the first hash value is the same as the second hash value.
11 . The apparatus of claim 1 wherein: the destination storage location comprises a cloud storage platform; and said at least one processing platform comprises a cloud tiering appliance.
12 . A method comprising:
computing, by executing one or more hashing algorithms, a first hash value for at least one file of a plurality of files to be tiered in connection with a file tiering operation of the plurality of files from a source storage location to a destination storage location, wherein computing the first hash value comprises computing a plurality of intermediate hash values for a plurality of data chunks of the at least one file, wherein the file tiering operation comprises writing the plurality of data chunks of the at least one file to an object on the destination storage location in response to computing the plurality of intermediate hash values;
comparing the first hash value to a second hash value for the at least one file, wherein the second hash value is computed from data in the object on the destination storage location corresponding to the at least one file;
determining, based at least in part on the comparing, whether the first hash value is the same as the second hash value, wherein the first hash value and the second hash value are determined to be the same when the first hash value and the second hash value match; and
proceeding with tiering of the at least one file in response to an affirmative determination;
wherein the method is performed by at least one processing platform comprising at least one processing device comprising a processor coupled to a memory.
13 . The method of claim 12 further comprising discontinuing the tiering of the at least one file in response to a negative determination.
14 . The method of claim 12 further comprising requesting the second hash value from a callback daemon.
15 . A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing platform causes said at least one processing platform:
to compute, by executing one or more hashing algorithms, a first hash value for at least one file of a plurality of files to be tiered in connection with a file tiering operation of the plurality of files from a source storage location to a destination storage location, wherein computing the first hash value comprises computing a plurality of intermediate hash values for a plurality of data chunks of the at least one file, wherein the file tiering operation comprises writing the plurality of data chunks of the at least one file to an object on the destination storage location in response to computing the plurality of intermediate hash values;
to compare the first hash value to a second hash value for the at least one file, wherein the second hash value is computed from data in the object on the destination storage location corresponding to the at least one file;
to determine, based at least in part on the comparing, whether the first hash value is the same as the second hash value, wherein the first hash value and the second hash value are determined to be the same when the first hash value and the second hash value match; and
to proceed with tiering of the at least one file in response to an affirmative determination.
16 . The computer program product of claim 15 wherein the program code further causes said at least one processing platform to request the second hash value from a callback daemon.
17 . The apparatus of claim 1 wherein said at least one processing platform is further configured:
in response to the affirmative determination, to cause generation of a stub file on the source storage location, wherein the stub file comprises an object identifier that indicates a location of the object on the destination storage location.
18 . The apparatus of claim 1 wherein the plurality of files is selected to be tiered based on one or more file tiering constraints.
19 . The apparatus of claim 18 wherein the one or more file tiering constraints comprise at least one of:
a time when a file was last accessed or modified;
a time when file attributes were last changed;
a size of a file;
a file name; and
a directory name.
20 . The computer program product of claim 15 wherein the plurality of files is selected to be tiered based on one or more file tiering constraints.