Data tiering based on data service status
Data service status information is obtained at a local storage tier regarding a data service operation performed on one or more data sets that qualify for remote storage based on a given policy. A determination is made whether or not to store at least a portion of the one or more data sets at a remote storage tier based on the obtained data service status information.
1. A method, comprising:
obtaining data service status information at a local storage tier regarding a data service operation performed on one or more data sets that qualify for storage at a remote storage tier based on a given policy, wherein the local storage tier and the remote storage tier are operatively coupled via a network;
determining whether or not to store at least a portion of the one or more data sets at the remote storage tier based on the obtained data service status information;
transferring a given one or more of the one or more data sets from the local storage tier to the remote storage tier over the network when a determination is made based on the data service status information that the data service operation has not been performed on the given one or more of the one or more data sets;
maintaining a given one or more of the one or more data sets at the local storage tier when a determination is made based on the data service status information that the data service operation has been performed on the given one or more of the one or more data sets; and
managing an index structure that keeps track of the one or more data sets that qualify for remote storage based on the given policy that bypass storage at the remote storage tier;
wherein the data service comprises data deduplication, the step of obtaining data service status information further comprises detecting a deduplication status for the one or more data sets, and the step of determining whether or not to store at least a portion of the one or more data sets at the remote storage tier further comprises evaluating whether or not at least a portion of the one or more data sets has already been deduplicated based on the detected deduplication status;
wherein, when at least one of the one or more data sets has not already been deduplicated, the transferring step further comprising reading the at least one data set from the local storage tier and writing the at least one read data set to the remote storage tier;
wherein memory locations of the one or more data sets at the primary storage tier are accessible in the index structure based on respective address offsets;
wherein data sets of the one or more data sets stored at the remote storage tier are recallable by deriving their address offsets from the address offsets stored for data sets at the primary storage tier in the index structure; and
wherein the steps are performed via one or more processing devices.
2. The method of claim 1 , wherein the local storage tier comprises primary storage of a local host device, and the remote storage tier comprises cloud storage of a cloud storage device remote from the local host device.
3. The method of claim 1 , wherein the one or more data sets comprise one or more data blocks.
4. The method of claim 1 , wherein, when at least one of the one or more data sets has already been deduplicated, the maintaining step further comprises bypassing storing the at least one data set at the remote storage tier and leaving the at least one data set stored at the local storage tier while storing the remaining data sets at the remote storage tier.
5. The method of claim 4 , further comprising reclaiming the storage capacity at the local storage tier for the data sets stored at the remote storage tier.
6. The method of claim 4 , further comprising reclaiming the storage capacity at the local storage tier based on the percentage of the one or more data sets that do not get stored at the remote storage tier.
7. The method of claim 1 , further comprising updating the index structure based on which data sets bypass storage at the remote storage tier.
8. The method of claim 1 , wherein the given policy comprises at least one criterion for determining whether or not the one or more data sets qualify for remote storage.
9. The method of claim 8 , wherein the at least one criterion is a determination that a data set is cold or inactive.
10. The method of claim 1 , further comprising marking a storage file or volume to which the one or more data sets belong to indicate which data sets have been stored at the remote storage tier and which data sets remain at the local storage tier, following the determining step.
11. The method of claim 1 wherein the index structure is stored on the primary storage tier.
12. A system, comprising:
at least one processor, coupled to a memory, and configured to:
obtain data service status information at a local storage tier regarding a data service operation performed on one or more data sets that qualify for storage at a remote storage tier based on a given policy, wherein the local storage tier and the remote storage tier are operatively coupled via a network;
determine whether or not to store at least a portion of the one or more data sets at the remote storage tier based on the obtained data service status information;
transfer a given one or more of the one or more data sets from the local storage tier to the remote storage tier over the network when a determination is made based on the data service status information that the data service operation has not been performed on the given one or more of the one or more data sets;
maintain a given one or more of the one or more data sets at the local storage tier when a determination is made based on the data service status information that the data service operation has been performed on the given one or more of the one or more data sets; and
manage an index structure that keeps track of the one or more data sets that qualify for remote storage based on the given policy that bypass storage at the remote storage tier;
wherein the data service comprises data deduplication, obtaining data service status information further comprises detecting a deduplication status for the one or more data sets, and determining whether or not to store at least a portion of the one or more data sets at the remote storage tier further comprises evaluating whether or not at least a portion of the one or more data sets has already been deduplicated based on the detected deduplication status;
wherein, when at least one of the one or more data sets has not already been deduplicated, the transferring step further comprising reading the at least one data set from the local storage tier and writing the at least one read data set to the remote storage tier;
wherein memory locations of the one or more data sets at the primary storage tier are accessible in the index structure based on respective address offsets; and
wherein data sets of the one or more data sets stored at the remote storage tier are recallable by deriving their address offsets from the address offsets stored for data sets at the primary storage tier in the index structure.
13. The system of claim 12 , wherein the index structure is stored on the primary storage tier.
14. The system of claim 12 , wherein when at least one of the one or more data sets has already been deduplicated, the at least one processor is further configured to:
bypass storing the at least one data set at the remote storage tier and leaving the at least one data set stored at the local storage tier while storing the remaining data sets at the remote storage tier; and
reclaim the storage capacity at the local storage tier for the data sets stored at the remote storage tier.
15. The system of claim 12 , wherein the at least one processor is further configured to:
update the index structure based on which data sets bypass storage at the remote storage tier.
16. An article of manufacture comprising a processor-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device implement the steps of:
obtaining data service status information at a local storage tier regarding a data service operation performed on one or more data sets that qualify for storage at a remote storage tier based on a given policy, wherein the local storage tier and the remote storage tier are operatively coupled via a network;
determining whether or not to store at least a portion of the one or more data sets at the remote storage tier based on the obtained data service status information;
transferring a given one or more of the one or more data sets from the local storage tier to the remote storage tier over the network when a determination is made based on the data service status information that the data service operation has not been performed on the given one or more of the one or more data sets;
maintaining a given one or more of the one or more data sets at the local storage tier when a determination is made based on the data service status information that the data service operation has been performed on the given one or more of the one or more data sets; and
managing an index structure that keeps track of the one or more data sets that qualify for remote storage based on the given policy that bypass storage at the remote storage tier;
wherein the data service comprises data deduplication, the step of obtaining data service status information further comprises detecting a deduplication status for the one or more data sets, and the step of determining whether or not to store at least a portion of the one or more data sets at the remote storage tier further comprises evaluating whether or not at least a portion of the one or more data sets has already been deduplicated based on the detected deduplication status;
wherein, when at least one of the one or more data sets has not already been deduplicated, the transferring step further comprising reading the at least one data set from the local storage tier and writing the at least one read data set to the remote storage tier;
wherein memory locations of the one or more data sets at the primary storage tier are accessible in the index structure based on respective address offsets; and
wherein data sets of the one or more data sets stored at the remote storage tier are recallable by deriving their address offsets from the address offsets stored for data sets at the primary storage tier in the index structure.
17. The article of manufacture of claim 16 , wherein the index structure is stored on the primary storage tier.
18. The article of manufacture of claim 16 , wherein when at least one of the one or more data sets has already been deduplicated, the code when executed by at least one processing device is further configured to implement the steps of:
bypassing storing the at least one data set at the remote storage tier and leaving the at least one data set stored at the local storage tier while storing the remaining data sets at the remote per QS storage tier; and
reclaim the storage capacity at the local storage tier for the data sets stored at the remote storage tier.
19. The article of manufacture of claim 16 , wherein the code when executed by at least one processing device is further configured to implement the step of:
updating the index structure based on which data sets bypass storage at the remote storage tier.
20. The article of manufacture of claim 16 , wherein the given policy comprises at least one criterion for determining whether or not the one or more data sets qualify for remote storage, and wherein the at least one criterion is a determination that a data set is cold or inactive.