Paging metadata into memory
One or more aspects of the present disclosure relate to paging metadata into memory. In embodiments, a metadata demand score is calculated for each extent of a storage array based on hit forecasts and read forecasts corresponding to input/output (IO) operations targeting each extent. Further, each extent can be ranked based on their respective metadata demand scores. Additionally, metadata can be paged into memory from a storage device of the storage array based on the ranking to reduce metadata page misses. For example, the memory can correspond to a global memory portion of the storage array.
1 . A method comprising:
calculating a metadata demand score for each extent of a storage array based on hit forecasts and read forecasts corresponding to input/output (IO) operations targeting each extent;
ranking each extent based on their respective metadata demand scores;
receiving extent level statistics from another storage array;
generating an effective metadata demand score for each extent of the storage array using the extent level statistics from the other storage and the metadata demand score for each extent of the storage array; and
paging metadata into memory from a storage device of the storage array based on the ranking to reduce metadata page misses, wherein the memory corresponds to a global memory portion of the storage array.
2 . The method of claim 1 , further comprising:
adjusting the metadata demand score for each extent based on a service level associated with each extent, with higher service levels resulting in a higher adjustment factor.
3 . The method of claim 1 , further comprising:
identifying Least Recently Used (LRU) cache slots corresponding to paged extent metadata in the memory;
identifying Most Recently Used (MRU) cache slots corresponding to the paged extent metadata in the memory; and
paging the metadata into the memory from the storage device based on the LRU cache slots and the MRU cache slots.
4 . The method of claim 1 , further comprising:
excluding each extent with a service level below a service level threshold from receiving the metadata demand score.
5 . The method of claim 1 , further comprising:
monitoring IO access patterns of each extent;
generating an activity mask for each extent based on the monitored IO access patterns; and
dynamically updating the metadata demand score for each extent using the activity mask for each extent.
6 . The method of claim 5 , further comprising:
establishing a metadata hint for a subject extent based on the metadata demand score of the subject extent and a service level of the extent.
7 . The method of claim 6 , further comprising:
inserting each extent with the metadata hint into the memory.
8 . The method of claim 7 , further comprising:
providing each extent with the metadata hint with an artificial timestamp to prioritize its retention in the memory based on predicted future access of each extent.
9 . The method of claim 8 , further comprising:
controlling a rate of metadata hinting for each extent based on the metadata demand score of each extent to avoid excessive churn in the memory.
10 . An apparatus with a memory and processor, the apparatus configured to:
calculate a metadata demand score for each extent of a storage array based on hit forecasts and read forecasts corresponding to input/output (IO) operations targeting each extent;
rank each extent based on their respective metadata demand scores;
receive extent level statistics from another storage array;
generate an effective metadata demand score for each extent of the storage array using the extent level statistics from the other storage and the metadata demand score for each extent of the storage array; and
page metadata into memory from a storage device of the storage array based on the ranking to reduce metadata page misses, wherein the memory corresponds to a global memory portion of the storage array.
11 . The apparatus of claim 10 , further configured to:
adjust the metadata demand score for each extent based on a service level associated with each extent, with higher service levels resulting in a higher adjustment factor.
12 . The apparatus of claim 10 , further configured to:
identify Least Recently Used (LRU) cache slots corresponding to paged extent metadata in the memory;
identify Most Recently Used (MRU) cache slots corresponding to the paged extent metadata in the memory; and
page the metadata into the memory from the storage device based on the LRU cache slots and the MRU cache slots.
13 . The apparatus of claim 10 , further configured to:
exclude each extent with a service level below a service level threshold from receiving the metadata demand score.
14 . The apparatus of claim 10 , further configured to:
monitor IO access patterns of each extent;
generate an activity mask for each extent based on the monitored IO access patterns; and
dynamically update the metadata demand score for each extent using the activity mask for each extent.
15 . The apparatus of claim 14 , further configured to:
establish a metadata hint for a subject extent based on the metadata demand score of the subject extent and a service level of the extent.
16 . The apparatus of claim 15 , further configured to:
insert each extent with the metadata hint into the memory.
17 . The apparatus of claim 16 , further configured to:
provide each extent with the metadata hint with an artificial timestamp to prioritize its retention in the memory based on predicted future access of each extent.
18 . The apparatus of claim 17 , further configured to:
control a rate of metadata hinting for each extent based on the metadata demand score of each extent to avoid excessive churn in the memory.