Metadata management in data storage systems
Embodiments of the present disclosure relate to an apparatus comprising a memory and at least one processor. The at least one processor is configured to: dynamically obtain a plurality of metadata from a global memory of a storage system; dynamically predict anticipated metadata based on the dynamically obtained metadata, wherein anticipated metadata is relevant to anticipated input/output (I/O) operations of the storage system; and dynamically instruct the storage system to load anticipated metadata into the global memory.
1. An apparatus comprising a memory and at least one processor configured to:
dynamically obtain a plurality of metadata from a global memory of a storage system;
dynamically predict anticipated metadata based on the dynamically obtained metadata, wherein anticipated metadata is relevant to anticipated input/output (I/O) operations of the storage system; and
dynamically predict anticipated metadata based on the dynamically obtained metadata, wherein anticipated metadata is relevant to anticipated input/output (I/O) operations of the storage system;
dynamically maintain and organize the plurality of metadata between the global memory and storage of an offload device based on the predicted anticipated metadata;
obtain a LUN map of one or more active LUNs during pre-determined periodic intervals; and
predict the LUNs that will be active during one or more future intervals based on the LUN map.
2. The apparatus of claim 1 , wherein each of the plurality of metadata includes one or more of the following information: a tag, allocation timer, holder, and lock.
3. The apparatus of claim 1 further configured to:
determine logical unit numbers (“LUNs”) of the one or more storage devices that are active during an interval based on the dynamically obtained metadata; and
predict LUNs of the one or more storage devices that will be active during one or more future intervals based on the active LUNs during the interval.
4. The apparatus of claim 1 further configured to ingest the one or more active LUN maps to predict the active LUNs during the one or more future intervals.
5. The apparatus of claim 3 further configured to:
determine LUN tracks of each LUN of the one or more storage devices associated with metadata stored in one or more cache slots of the one or more storage devices that are active during an interval; and
determine a set of LUN tracks for each of the LUNs of the one or more storage devices that are correlated, wherein LUN tracks are correlated when the LUN tracks are needed for I/O operations when each LUN track's corresponding LUN is active during the interval.
6. The apparatus of claim 5 further configured to:
obtain active track maps of the active LUN tracks of each LUN during pre-determined periodic intervals; and
determine the set of LUN tracks for each of the LUNs that are correlated using convolution processing layers.
7. The apparatus of claim 6 further configured to predict a set of LUN tracks needed for I/O operations during the one or more future intervals using the prediction of the active LUNs and the determined set of correlated LUN tracks.
8. The apparatus of claim 7 further configured to predict the set of LUN tracks needed for I/O operations during the one or more future intervals using a pattern of metadata present in the one or more cache slots of the one or more storage devices during a current interval.
9. The apparatus of claim 7 further configured to:
generate an error score for the prediction of the set of LUN tracks needed for I/O operations during the one or more future intervals; and
in response to determining the error score exceeds a threshold, issue a training request to a classifier, wherein the dynamically predicted anticipated metadata is further based on error data used to generate the error score.
10. A method comprising:
dynamically obtaining a plurality of metadata from a global memory of a storage system;
dynamically predicting anticipated metadata based on the dynamically obtained metadata, wherein anticipated metadata is relevant to anticipated input/output (I/O) operations of the storage system;
dynamically maintaining and organizing the plurality of metadata between the global memory and storage of an offload device based on the predicted anticipated metadata;
obtaining a LUN map of one or more active LUNs during pre-determined periodic intervals; and
predicting the LUNs that will be active during one or more future intervals based on the LUN map.
11. The method of claim 10 , wherein each of the plurality of metadata includes one or more of the following information: a tag, allocation timer, holder, and lock.
12. The method of claim 10 further comprising:
determining logical unit numbers (“LUNs”) of the one or more storage devices that are active during an interval based on the dynamically obtained metadata; and
predicting LUNs of the one or more storage devices that will be active during one or more future intervals based on the active LUNs during the interval.
13. The method of claim 10 further comprising ingesting the one or more active LUN maps to predict the active LUNs during the one or more future intervals.
14. The method of claim 12 further comprising:
determining LUN tracks of each LUN of the one or more storage devices associated with metadata stored in one or more cache slots of the one or more storage devices that are active during an interval; and
determining a set of LUN tracks for each of the LUNs of the one or more storage devices that are correlated, wherein LUN tracks are correlated when the LUN tracks are needed for I/O operations when each LUN track's corresponding LUN is active during the interval.
15. The method of claim 14 further comprising:
obtaining active track maps of the active LUN tracks of each LUN during pre-determined periodic intervals; and
determining the set of LUN tracks for each of the LUNs that are correlated using convolution processing layers.
16. The method of claim 15 further comprising predicting a set of LUN tracks needed for I/O operations during the one or more future intervals using the prediction of the active LUNs and the determined set of correlated LUN tracks.
17. The method of claim 16 further comprising predicting the set of LUN tracks needed for I/O operations during the one or more future intervals using a pattern of metadata present in the one or more cache slots of the one or more storage devices during a current interval.
18. The method of claim 16 further comprising:
generating an error score for the prediction of the set of LUN tracks needed for I/O operations during the one or more future intervals; and
in response to determining the error score exceeds a threshold, issuing a training request to a classifier, wherein the dynamically predicted anticipated metadata is further based on error data used to generate the error score.