System and method for dynamic data migration
A storage gateway for managing data includes a persistent storage and a processor. The persistent storage stores a stored data history map. The processor obtains a data storage request for a datum from a client; stores the datum in a first storage based on a quality of storage service requirement associated with the datum; after the datum is stored in the first storage, updates the stored data history map associated with the datum based on the quality of storage service characteristics of the first storage; and after updating the stored data history map, migrates the datum to a second storage based on the quality of storage service requirement and the updated stored data history map.
1. A storage gateway for managing data, comprising:
a persistent storage comprising a stored data history map;
a processor programmed to:
obtain a data storage request for a datum from a client;
store the datum in a first storage based on a quality of storage service requirement associated with the datum;
after the datum is stored in the first storage, update the stored data history map associated with the datum based on the quality of storage service characteristics of the first storage, wherein the stored data history map specifies:
storage locations of where the datum is stored over time, and
an actual quality of storage service provided by the storage locations; and
after updating the stored data history map, migrate the datum to a second storage based on the quality of storage service requirement and the updated stored data history map.
2. The storage gateway of claim 1 , wherein the second storage is not available to store the datum when the datum is stored in the first storage.
3. The storage gateway of claim 1 , wherein the quality of storage service requirement specifies a maximum amount of time the storage gateway is allowed to provide the datum in response to a client request.
4. The storage gateway of claim 1 , wherein the quality of storage service requirement specifies a plurality of predetermined periods of time, wherein each predetermined period of time of the plurality of predetermined periods of time is associated with a respective quality of storage service during the predetermined period of time.
5. The storage gateway of claim 4 , wherein each respective quality of storage service specifies a maximum amount of time the storage gateway is allowed to provide the datum in response to a client request during an associated predetermined period of time.
6. The storage gateway of claim 5 , wherein a first respective quality of storage service and a second respective quality of storage service specify different maximum amounts of time.
7. The storage gateway of claim 1 , wherein migrating the datum to the second storage based on the quality of storage service requirement and the updated stored data history map comprises:
identifying a required quality of storage service at a time of the migrating based on the quality of storage service requirement;
matching the required quality of storage service to respective quality of storage services provided by a plurality of storages, wherein each storage of the plurality of storages are operably connected to the storage gateway at the time of migrating; and
making a copy of the datum in the second storage,
wherein the second storage is a storage of the plurality of storages.
8. The storage gateway of claim 7 , wherein identifying the required quality of storage service at the time of the migrating based on the quality of storage service requirement comprises:
identifying a storage life of the datum based on the updated stored data history map;
matching the identified storage life to an entry of the quality of storage service requirements; and
using the required quality of storage service specified by the entry as the identified required quality of storage service,
wherein the second storage is a storage of the plurality of storages.
9. The storage gateway of claim 7 , wherein matching the required quality of storage service to respective quality of storage services provided by a plurality of storages comprises:
extracting a quality of storage service metric from the required quality of storage service;
comparing the extracted quality of storage service metric to respective quality of storage services provided by all storages operably connected to the storage gateway at the time of migrating; and
identifying the matched plurality of storages based on the comparing.
10. The storage gateway of claim 9 , wherein matching the required quality of storage service to respective quality of storage services provided by a plurality of storages further comprises:
extracting a second quality of storage service metric from the required quality of storage service;
comparing the second extracted quality of storage service metric to respective quality of storage services provided by the matched plurality of storages; and
reducing a cardinality of the matched plurality of storages based on the comparing.
11. The storage gateway of claim 1 , wherein updating the stored data history map associated with the datum based on the quality of storage service characteristics of the first storage comprises:
determining an average amount of time the first storage takes to return data stored in the first storage in response to data access requests at the time the datum is stored in the first storage;
identifying an entry of the quality of storage service characteristics associated with the first storage; and
updating the entry to reflect the determined average amount of time at the time the datum is stored.
12. The storage gateway of claim 1 , wherein updating the stored data history map associated with the datum based on the quality of storage service characteristics of the first storage comprises:
determining an average amount of time the first storage takes to return data stored in the first storage in response to data access requests over a predetermined amount of time;
identifying an entry of the quality of storage service characteristics associated with the first storage; and
updating the entry to reflect the determined average amount of time at the time the datum is stored.
13. The storage gateway of claim 12 , wherein the predetermined amount of time comprises a period of time preceding the time the datum is stored in the first storage.
14. The storage gateway of claim 1 , wherein the updated stored data history map specifies an amount of time the datum was stored in a cache of the storage gateway before being stored in the first storage.
15. A method of operating a storage gateway for managing data, comprising:
obtaining a data storage request for a datum from a client;
storing the datum in a first storage based on a quality of storage service requirement associated with the data;
after storing the datum, updating a stored data history map associated with the datum based on the quality of storage service characteristics of the first storage, wherein the stored data history map specifies:
storage locations of where the datum is stored over time, and
an actual quality of storage service provided by the storage locations; and
after updating the stored data history map, migrating the datum to a second storage based on the quality of storage service requirement and the updated stored data history map.
16. The method of claim 15 , wherein migrating the datum to the second storage based on the quality of storage service requirement and the updated stored data history map comprises:
identifying a required quality of storage service at a time of the migrating based on the quality of storage service requirement;
matching the required quality of storage service to respective quality of storage services provided by a plurality of storages, wherein each storage of the plurality of storages are operably connected to the storage gateway at the time of migrating; and
making a copy of the datum in the second storage,
wherein the second storage is a storage of the plurality of storages.
17. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for operating a storage gateway for managing data, the method comprising:
obtaining a data storage request for a datum from a client;
storing the datum in a first storage based on a quality of storage service requirement associated with the data;
after storing the datum, updating a stored data history map associated with the datum based on the quality of storage service characteristics of the first storage, wherein the stored data history map specifies:
storage locations of where the datum is stored over time, and
an actual quality of storage service provided by the storage locations; and
after updating the stored data history map, migrating the datum to a second storage based on the quality of storage service requirement and the updated stored data history map.
18. The non-transitory computer readable medium of claim 17 , wherein migrating the datum to the second storage based on the quality of storage service requirement and the updated stored data history map comprises:
identifying a required quality of storage service at a time of the migrating based on the quality of storage service requirement;
matching the required quality of storage service to respective quality of storage services provided by a plurality of storages, wherein each storage of the plurality of storages are operably connected to the storage gateway at the time of migrating; and
making a copy of the datum in the second storage,
wherein the second storage is a storage of the plurality of storages.