Hybrid file system for virtual machine storage
A data storage system is disclosed, including: a first storage device of a first storage device type, a second storage device of a second storage device type, and a processor configured to implement a hybrid file system configured to store each of a plurality of data values in at least a corresponding primary location, which for a given data value may be in either the first storage device or the second storage device; wherein the hybrid file system is configured to use a first access approach optimized based on a first access attribute of the first storage device type to interact with the first storage device and a second access approach optimized based on a second access attribute of the second storage device type to interact with the second storage device. Reading data is disclosed, including: determining that a storage location on a first tier storage from which a requested data was written back to a second tier storage has not been reclaimed by a garbage collection associated with the first tier storage, and retrieving the data from the storage location on the first tier storage.
1. A data storage system, comprising:
a first storage device of a first storage device type;
a second storage device of a second storage device type; and
a processor configured to implement a hybrid file system configured to store each of a plurality of data values in at least a corresponding primary location, which for a given data value may be in at least one of the first storage device or the second storage device;
wherein the hybrid file system is configured to use a first access approach optimized based on a first access attribute of the first storage device type to interact with the first storage device and a second access approach optimized based on a second access attribute of the second storage device type to interact with the second storage device, wherein data comprising a set of one or more blocks of a first size associated with the first storage device is configured to be written back to the second storage device, wherein the set of one or more blocks of the first size associated with the first storage device to be written back corresponds to a second size of a block associated with the second storage device.
2. The data storage system of claim 1 , wherein the first access approach comprises a log-structured approach.
3. The data storage system of claim 1 , wherein the first access approach comprises storing data in blocks of the first size optimized for the first storage device type and the second access approach comprises writing data back from the first storage device to the second storage device in blocks of the second size optimized for the second storage device type.
4. The data storage system of claim 3 , wherein the processor is further configured to determine whether a condition for writing back is met.
5. The data storage system of claim 4 , wherein the condition for writing back is based at least in part on falling below a threshold of access counts associated with data at the first storage device.
6. The data storage system of claim 4 , wherein the condition for writing back is based at least in part on exceeding a quota of an amount of storage at the first storage device.
7. The data storage system of claim 1 , wherein the at least a corresponding primary location in either the first storage device or the second storage device for a given data value is indicated in metadata.
8. The data storage system of claim 7 , wherein the metadata includes a data structure that includes for each of a plurality of portions of a data object a corresponding storage location configured to store data indicating a location in which data comprising the portion is stored.
9. The data storage system of claim 7 , wherein, subsequent to a write back operation, the metadata is updated to indicate for a data written back from a location in the first storage device a corresponding location to which the data has been written in the second storage device.
10. The data storage of claim 9 , wherein the data structure includes a metadata storage location configured to store a location data indicating a location in the first storage device in which the written back data remains stored, notwithstanding the data having been written back to the second storage device.
11. The data storage of claim 9 , wherein the data written back to the second storage device remains stored in the first storage device until a reclamation criterion is met.
12. The data storage of claim 11 , wherein the processor is further configured to delete or otherwise update the location data upon reclamation of the location in the first storage device.
13. The data storage of claim 7 , wherein the metadata includes access counts for data stored at the first storage device.