Selectively delaying cache flushing to promote write efficiency
A technique for managing storage in a data storage system includes ingesting host data into a data log backed by pages in a cache. The host data are addressed to specified locations in a data object stored in persistent storage devices. When mapping pages in cache to respective target locations in the persistent storage devices, a flag is selectively asserted to indicate whether there is contiguous free space at the target locations. The cache responds to the flag by delaying flushing of flagged pages, which provides more time for the cache to aggregate pages that map to contiguous free space. Meanwhile, pages that are not flagged can be flushed more quickly, enabling the cache to operate efficiently even though flushing of some pages is delayed.
1. A method of managing write requests in a data storage system, the method comprising:
receiving IO (Input/Output) requests specifying writes of host data to identified portions of a data object;
ingesting the host data into a data log, the data log storing the host data in a set of pages of a cache;
performing a rename operation on each of the set of pages in the cache, each rename operation (i) identifying a target location in persistent storage to which that page will be flushed and (ii) providing a coalesce-expected flag for that page to indicate whether the persistent storage has contiguous free space at the target location; and
for each of the set of pages for which the coalesce-expected flag is asserted, processing that page for delayed flushing from the cache to the respective target location, the delayed flushing providing more time prior to flushing for the cache to aggregate contiguous writes than would be available if the page were processed for less-delayed flushing.
2. The method of claim 1 , further comprising, for each of the set of pages for which the coalesce-expected flag is not asserted, processing that page for less-delayed flushing from the cache to the respective target location.
3. The method of claim 2 , further comprising:
aggregating a predetermined number of pages for which the coalesce-expected flag is asserted and for which respective rename operations identify target locations in the persistent storage that are contiguous; and
once the predetermined number of pages have been aggregated, processing each of the aggregated pages going forward for less-delayed flushing from the cache to the respective target locations.
4. The method of claim 3 ,
wherein the cache employs a flushing policy that directs the cache to flush pages based on age, such that the cache flushes older pages before it flushes younger pages, and
wherein processing a page for delayed flushing causes the page to age at a slower rate whereas processing a page for less-delayed flushing causes the page to age at a faster rate.
5. The method of claim 4 ,
wherein the data storage system stores the data object in a file system, the file system having a physical address space backed by the persistent storage, and
wherein each of a set of ranges of contiguous physical addresses in the file system is backed by a respective range of contiguous storage locations in the persistent storage.
6. The method of claim 4 , wherein identifying the target location to which each of the set of pages will be flushed includes mapping a logical address to which host data in that page is directed to a corresponding physical address in the file system at which that host data will be placed.
7. The method of claim 6 ,
wherein at least a portion of the file system is rendered as a set of windows, each window including a range of contiguous physical addresses in the file system backed by a corresponding range of contiguous storage locations in the persistent storage, and
wherein providing the coalesce-expected flag for a page includes asserting the coalesce-expected flag in response to the physical address in the file system to which that page is mapped residing within a window that has free space.
8. The method of claim 7 ,
wherein the cache arranges pages into groups using reference structures, each reference structure pointing to multiple pages and aligning with a respective range of contiguous physical address space in the file system, and
wherein the method further includes flushing pages pointed to by a reference structure by flushing all pages pointed to by that reference structure together at one time.
9. The method of claim 8 , wherein a reference structure is made to point to a cache page in response to performing a rename operation on that cache page that identifies a mapped location in the file system to which that reference structure is aligned, and wherein the reference structure is made to point to additional cache pages over time as additional rename operations map pages to other physical addresses to which that reference structure is aligned.
10. A data storage system, comprising control circuitry that includes a set of processing units coupled to memory, the control circuitry constructed and arranged to:
receive IO (Input/Output) requests specifying writes of host data to identified portions of a data object;
ingest the host data into a data log, the data log storing the host data in a set of pages of a cache;
perform a rename operation on each of the set of pages in the cache, each rename operation (i) identifying a target location in persistent storage to which that page will be flushed and (ii) providing a coalesce-expected flag for that page to indicate whether the persistent storage has contiguous free space at the target location; and
for each of the set of pages for which the coalesce-expected flag is asserted, processing that page for delayed flushing from the cache to the respective target location, the delayed flushing providing more time prior to flushing for the cache to aggregate contiguous writes than would be available if the page were processed for less-delayed flushing.
11. The data storage system of claim 10 , wherein the control circuitry is further constructed and arranged to:
aggregate a predetermined number of pages for which the coalesce-expected flag is asserted and for which respective rename operations identify target locations in the persistent storage that are contiguous; and
once the predetermined number of pages have been aggregated, process each of the aggregated pages going forward for less-delayed flushing from the cache to the respective target locations.
12. A computer program product including a set of non-transitory, computer-readable media having instructions which, when executed by control circuitry of a data storage system, cause the control circuitry to perform a method for managing write requests, the method comprising:
receiving IO (Input/Output) requests specifying writes of host data to identified portions of a data object;
ingesting the host data into a data log, the data log storing the host data in a set of pages of a cache;
performing a rename operation on each of the set of pages in the cache, each rename operation (i) identifying a target location in persistent storage to which that page will be flushed and (ii) providing a coalesce-expected flag for that page to indicate whether the persistent storage has contiguous free space at the target location; and
for each of the set of pages for which the coalesce-expected flag is asserted, processing that page for delayed flushing from the cache to the respective target location, the delayed flushing providing more time prior to flushing for the cache to aggregate contiguous writes than would be available if the page were processed for less-delayed flushing.
13. The computer program product of claim 12 , wherein the method further comprises, for each of the set of pages for which the coalesce-expected flag is not asserted, processing that page for less-delayed flushing from the cache to the respective target location.
14. The computer program product of claim 13 , wherein the method further comprises:
aggregating a predetermined number of pages for which the coalesce-expected flag is asserted and for which respective rename operations identify target locations in the persistent storage that are contiguous; and
once the predetermined number of pages have been aggregated, processing each of the aggregated pages going forward for less-delayed flushing from the cache to the respective target locations.
15. The computer program product of claim 14 ,
wherein the cache employs a flushing policy that directs the cache to flush pages based on age, such that the cache flushes older pages before it flushes younger pages, and
wherein processing a page for delayed flushing causes the page to age at a slower rate whereas processing a page for less-delayed flushing causes the page to age at a faster rate.
16. The computer program product of claim 15 ,
wherein the data storage system stores the data object in a file system, the file system having a physical address space backed by the persistent storage, and
wherein each of a set of ranges of contiguous physical addresses in the file system is backed by a respective range of contiguous storage locations in the persistent storage.
17. The computer program product of claim 15 , wherein identifying the target location to which each of the set of pages will be flushed includes mapping a logical address to which host data in that page is directed to a corresponding physical address in the file system at which that host data will be placed.
18. The computer program product of claim 17 ,
wherein at least a portion of the file system is rendered as a set of windows, each window including a range of contiguous physical addresses in the file system backed by a corresponding range of contiguous storage locations in the persistent storage, and
wherein providing the coalesce-expected flag for a page includes asserting the coalesce-expected flag in response to the physical address in the file system to which that page is mapped residing within a window that has free space.
19. The computer program product of claim 18 ,
wherein the cache arranges pages into groups using reference structures, each reference structure pointing to multiple pages and aligning with a respective range of contiguous physical address space in the file system, and
wherein the method further includes flushing pages pointed to by a reference structure by flushing all pages pointed to by that reference structure together at one time.
20. The computer program product of claim 19 , wherein a reference structure is made to point to a cache page in response to performing a rename operation on that cache page that identifies a mapped location in the file system to which that reference structure is aligned, and wherein the reference structure is made to point to additional cache pages over time as additional rename operations map pages to other physical addresses to which that reference structure is aligned.