Time series based MDP paging for snapshot, replication and migration
Metadata pages used exclusively for maintenance IOs associated with snapshots, data migration, or data replication are paged-out according to a time series model forecasted TTL. Metadata pages used for production IOs are paged-out using an LRU algorithm. The forecasted TTL corresponds to the expected time during which the metadata pages will be needed to service the maintenance IOs.
1 . A method implemented by a storage array having compute nodes with volatile memory, the compute nodes managing access to non-volatile disk storage with paged metadata, the method comprising:
identifying an input-output operation (IO) between the non-volatile disk storage and the volatile memory of one of the compute nodes as being a maintenance IO for snapshot generation, data migration, or remote replication of storage objects;
paging-in a first metadata page corresponding to the maintenance IO from the non-volatile disk storage to the volatile memory of one of the compute nodes; and
paging-out the first metadata page after a forecasted time-to-live (TTL) has expired and before the first metadata page becomes a least recently used metadata page,
whereby metadata pages used exclusively for maintenance IOs are evicted after less time in the volatile memory than metadata pages not used exclusively for maintenance IOs.
2 . The method of claim 1 further comprising using a time series model to forecast the TTL.
3 . The method of claim 2 further comprising identifying an IO as being a production IO, paging-in a second metadata page corresponding to the production IO from non-volatile disk storage to memory, and paging-out the second metadata page based on selection by a least recently used (LRU) algorithm.
4 . The method of claim 3 further comprising identifying a third page of metadata as being used for both production IOs and maintenance IOs and paging-out the third metadata page based on selection by the LRU algorithm.
5 . The method of claim 4 further comprising applying a weight to the forecasted TTL.
6 . The method of claim 5 further comprising applying a weight to a fall-through time (FTT) determined by the LRU algorithm.
7 . The method of claim 6 further comprising combining the weighted forecasted TTL and the weighted LRU FTT.
8 . An apparatus, comprising:
a plurality of disk drives; and
at least one compute node configured to manage access to the disk drives, the compute node comprising a processor, memory, and a metadata paging manager configured to:
identify an input-output operation (IO) between the disk drives and the memory of one of the compute nodes as being a maintenance IO for snapshot generation, data migration, or remote replication of storage objects;
page-in a first metadata page corresponding to the maintenance IO from the disk drives to the memory of one of the compute nodes; and
page-out the first metadata page after a forecasted time-to-live (TTL) has expired and before the first metadata page becomes a least recently used metadata page,
whereby metadata pages used exclusively for maintenance IOs are evicted after less time in the volatile memory than metadata pages not used exclusively for maintenance IOs.
9 . The apparatus of claim 8 further comprising the metadata paging manager configured to use a time series model to forecast the TTL.
10 . The apparatus of claim 9 further comprising the metadata paging manager configured to identify an IO as being a production IO, page-in a second metadata page corresponding to the production IO from non-volatile disk storage to memory, and page-out the second metadata page based on selection by a least recently used (LRU) algorithm.
11 . The apparatus of claim 10 further comprising the metadata paging manager configured to identify a third page of metadata as being used for both production IOs and maintenance IOs and page-out the third metadata page based on selection by the LRU algorithm.
12 . The apparatus of claim 11 further comprising the metadata paging manager configured to apply a weight to the forecasted TTL.
13 . The apparatus of claim 12 further comprising the metadata paging manager configured to apply a weight to a fall-through time (FTT) determined by the LRU algorithm.
14 . The apparatus of claim 13 further comprising the metadata paging manager configured to combine the weighted forecasted TTL and the weighted LRU FTT.
15 . A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method implemented by a storage array having compute nodes with volatile memory, the compute nodes managing access to non-volatile disk storage with paged metadata, the method comprising:
identifying an input-output operation (IO) between the non-volatile disk storage and the volatile memory of one of the compute nodes as being a maintenance IO for snapshot generation, data migration, or remote replication of storage objects;
paging-in a first metadata page corresponding to the maintenance IO from the non-volatile disk storage to the volatile memory of one of the compute nodes; and
paging-out the first metadata page after a forecasted time-to-live (TTL) has expired and before the first metadata page becomes a least recently used metadata page,
whereby metadata pages used exclusively for maintenance IOs are evicted after less time in the volatile memory than metadata pages not used exclusively for maintenance IOs.
16 . The non-transitory computer-readable storage medium of claim 15 in which the method further comprises using a time series model to forecast the TTL.
17 . The non-transitory computer-readable storage medium of claim 16 in which the method further comprises identifying an IO as being a production IO, paging-in a second metadata page corresponding to the production IO from non-volatile disk storage to memory, and paging-out the second metadata page based on selection by a least recently used (LRU) algorithm.
18 . The non-transitory computer-readable storage medium of claim 17 in which the method further comprises identifying a third page of metadata as being used for both production IOs and maintenance IOs and paging-out the third metadata page based on selection by the LRU algorithm.
19 . The non-transitory computer-readable storage medium of claim 18 in which the method further comprises applying a weight to the forecasted TTL and applying a weight to a fall-through time (FTT) determined by the LRU algorithm.
20 . The non-transitory computer-readable storage medium of claim 19 in which the method further comprises combining the weighted forecasted TTL and the weighted LRU FTT.