Hybrid data tiering with cloud storage
Hybrid data tiering, including: replicating multiple data objects from a first storage location on a first computer system to a second storage location at a second computer system; receiving, from a client computing system, a retention policy specifying one or more rules or conditions to evaluate to determine whether to continue storing data at a first storage location in addition to storing the data at a second storage location; identifying, in accordance with the retention policy, some of the multiple data objects to continue storing at the first storage location, wherein all of the multiple data objects remain stored at the second storage location; and selecting the identified some of the multiple data objects to remain stored at the first storage location.
1. A method comprising:
replicating multiple data objects from a first storage location on a first computer system to a second storage location at a second computer system;
receiving, from a client computing system, a retention policy identifying characteristics of data objects to continue storing at the first storage location in addition to storing the data at the second storage location, wherein the retention policy specifies that a greater number of references by applications to a given object corresponds to a higher priority for the given object remaining stored at the first storage location; and
selecting, in accordance with the retention policy, some of the multiple data objects to continue storing at the first storage location, wherein all of the multiple data objects remain stored at the second storage location.
2. The method of claim 1 , wherein, in response to a request for one of the multiple data objects not stored at the first storage location, the one of the multiple data objects not stored at the first storage location is retrieved from the second storage location.
3. The method of claim 2 , wherein the retention policy specifies data backup information, and wherein the client computing system comprises a user-level management layer that specifies the data backup information.
4. The method of claim 1 , wherein the first storage location is part of a storage system at a first geographic location, and wherein the second storage location is implemented within a remote, cloud services provider.
5. The method of claim 4 , wherein the cloud services provider implements the second storage location as data object store or block storage.
6. The method of claim 1 , wherein the first storage location is part of a local storage system at a first geographic location, and wherein the second storage location is implemented within a remote storage system.
7. The method of claim 1 , wherein the retention policy is based upon a historical pattern of storage access from one or more processes operating on storage at the first storage location.
8. The method of claim 1 , wherein the retention policy specifies different retention behavior for different users, or wherein the retention policy specifies different retention behavior for different priority processes operating at the first storage location.
9. The method of claim 1 , wherein the selected data objects to remain stored at the first storage location correspond to a data backup, and wherein the retention policy specifies a length of time for the selected data objects to remain stored at the first storage location based on an expected window of time within which data backup is expected to be referenced, and wherein the data objects among the multiple data objects not selected to remain stored at the first storage location are deleted.
10. The method of claim 1 , wherein one or more given data objects not selected to remain stored at the first storage location are not removed until after the one or more given data objects are successfully stored at the second storage location.
11. An apparatus comprising:
one or more hardware processors;
one or more data storage resources; and
a hybrid data tiering controller module configured to:
replicate multiple data objects from a first storage location on a first computer system to a second storage location at a second computer system, wherein the multiple data objects are received from a client computing system;
receive, from a client computing system, a retention policy identifying characteristics of data objects to continue storing at the first storage location in addition to storing the data at the second storage location, wherein the retention policy specifies that a greater number of references by applications to a given object corresponds to a higher priority for the given object remaining stored at the first storage location; and
select, in accordance with the retention policy, some of the multiple data objects to continue storing at the first storage location, wherein all of the multiple data objects remain stored at the second storage location.
12. The apparatus of claim 11 , wherein the first storage location is part of a storage system at a first geographic location, and wherein the second storage location is implemented within a remote, cloud services provider.
13. The apparatus of claim 11 , wherein the cloud services provider implements the second storage location as data object store or block storage.
14. The apparatus of claim 11 , wherein the first storage location is part of a local storage system at a first geographic location, and wherein the second storage location is implemented within a remote storage system.
15. The apparatus of claim 11 , wherein the retention policy specifies different retention behavior for different users, or wherein the retention policy specifies different retention behavior for different priority processes operating at the first storage location.
16. The apparatus of claim 11 , wherein the selected data objects to remain stored at the first storage location correspond to a data backup, and wherein the retention policy specifies a length of time for the selected data objects to remain stored at the first storage location based on an expected window of time within which data backup is expected to be referenced, and wherein the data objects among the multiple data objects not selected to remain stored at the first storage location are deleted.
17. A computer program product disposed upon a computer readable medium, the computer program product comprising computer program instructions that, when executed, cause a computer to carry out the steps of:
replicating multiple data objects from a first storage location on a first computer system to a second storage location at a second computer system, wherein the multiple data objects are received from a client computing system;
receiving, from a client computing system, a retention policy identifying characteristics of data objects to continue storing at the first storage location in addition to storing the data at the second storage location, wherein the retention policy specifies that a greater number of references by applications to a given object corresponds to a higher priority for the given object remaining stored at the first storage location; and
selecting, in accordance with the retention policy, some of the multiple data objects to continue storing at the first storage location, wherein all of the multiple data objects remain stored at the second storage location.