Caching techniques
Techniques provide for specifying priorities and thresholds used for cache promotion and flushing on a per pool basis. The thresholds may include a promotion threshold and a flushing threshold. The promotion threshold is used to determine whether data portions of the pool are promoted to a secondary cache. The flushing threshold is used to determine whether any data portions of the pool currently stored in the secondary cache are flushed to non-volatile physical storage. Each of the promotion threshold and the flushing threshold for the pool may be determined in accordance with the priority specified for the pool. For remaining data portions that are included in a pool for which a pool specific priority is not specified, a default technique may be used, for example, which uses the same promotion threshold and the same flushing threshold for all pools for which a pool specific priority is not specified.
1. A method of caching data comprising:
performing promotion processing to promote one or more data portions to a secondary cache, wherein said secondary cache and a primary data cache are included in a plurality of caching layers, said promotion processing using a first promotion threshold specified for a first pool of logical devices to determine whether a first data portion of the first pool of logical devices is promoted to the secondary cache, said promotion processing using a second promotion threshold specified for a second pool of logical devices to determine whether a second data portion of the second pool of logical devices is promoted to the secondary cache;
determining whether a current number of free cache pages in the secondary cache is greater than a specified minimum number of secondary cache pages; and
responsive to determining the current number of free cache pages in the secondary cache is not greater than the specified minimum number of free pages, performing flushing processing to flush one or more data portions from the secondary cache to physical storage, said flushing processing using a first flushing threshold specified for the first pool of logical devices to determine whether a third data portion of the first pool of logical devices is flushed from the secondary cache, wherein the first flushing threshold denotes a maximum number of secondary cache pages used for storing data of each logical device of the first pool and the flushing processing applies the first flushing threshold individually to each logical device of the first pool, said flushing processing using a second flushing threshold specified for the second pool of logical devices to determine whether a fourth data portion of the second pool of logical devices is flushed from the secondary cache, wherein the second flushing threshold denotes a maximum number of secondary cache pages used for storing data of each logical device of the second pool and the flushing processing applies the second flushing threshold individually to each logical device of the second pool.
2. The method of claim 1 , wherein the promotion processing includes:
determining whether a first count, denoting a number of I/Os referencing the first data portion, is equal to or greater than the first promotion threshold; and
responsive to determining the first count is equal to or greater than the first promotion threshold, promoting the first data portion to the secondary cache.
3. The method of claim 2 , wherein the promotion processing includes:
determining whether a second count, denoting a number of I/Os referencing the second data portion, is equal to or greater than the second promotion threshold; and
responsive to determining the second count is equal to or greater than the second promotion threshold, promoting the second data portion to the secondary cache.
4. The method of claim 1 , wherein the third data portion is stored at a first logical address of a first logical device, said first logical device being included in the first pool, and wherein the flushing processing includes:
determining whether a third count, denoting a number of cache pages of the secondary cache include cached data of the first logical device, exceeds the first flushing threshold; and
responsive to determining the third count exceeds the first flushing threshold, flushing the third data portion from the secondary cache to physical storage.
5. The method of claim 4 , wherein data portions of the first logical device are flushed from the secondary cache to physical storage until the third count is below the first flushing threshold.
6. The method of claim 5 , wherein the fourth data portion is stored at a second logical address of a second logical device, said second logical device being included in the second pool, and wherein the flushing processing includes:
determining whether a fourth count, denoting a number of cache pages of the secondary cache include cached data of the second logical device, exceeds the second flushing threshold; and
responsive to determining the fourth count exceeds the second flushing threshold, flushing the fourth data portion from the secondary cache to physical storage.
7. The method of claim 6 , wherein data portions of the second logical device are flushed from the secondary cache to physical storage until the fourth count is below the second flushing threshold.
8. The method of claim 1 , further comprising:
receiving a first set of values for the first pool, said first set of values including a first priority for the first pool, the first flushing threshold for the first pool and the first promotion threshold for the first pool; and
receiving a second set of values for the second pool, said second set of values including a second priority for the second pool, the second flushing threshold for the second pool and the second promotion threshold for the second pool.
9. The method of claim 8 , wherein the first flushing threshold and the first promotion threshold are calculated using the first priority.
10. The method of claim 9 , wherein the second flushing threshold and the second promotion threshold are calculated using the second priority.
11. The method of claim 8 , wherein said first pool includes a first plurality of logical devices and the second pool includes a second plurality of logical devices, wherein the first set of values are used in connection with performing promotion and flushing of the secondary cache with respect to data portions of the first plurality of logical devices, and wherein the second set of values are used in connection with performing promotion and flushing of the secondary cache with respect to data portions of the second plurality of logical devices.
12. The method of claim 1 , wherein the flushing processing is performed responsive to determining that a current number of free pages in the secondary cache is below a specified minimum threshold.
13. The method of claim 1 , wherein the secondary cache includes a plurality of flash-based storage devices.
14. The method of claim 1 , wherein the plurality of caching layers are included in a data storage system.
15. A system comprising:
one or more processors; and
a memory comprising code stored therein that, when executed by at least one of the processors, performs a method of caching data comprising:
performing promotion processing to promote one or more data portions to a secondary cache, wherein said secondary cache and a primary data cache are included in a plurality of caching layers, said promotion processing using a first promotion threshold specified for a first pool of logical devices to determine whether a first data portion of the first pool of logical devices is promoted to the secondary cache, said promotion processing using a second promotion threshold specified for a second pool of logical devices to determine whether a second data portion of the second pool of logical devices is promoted to the secondary cache;
determining whether a current number of free cache pages in the secondary cache is greater than a specified minimum number of secondary cache pages; and
responsive to determining the current number of free cache pages in the secondary cache is not greater than the specified minimum number of free pages, performing flushing processing to flush one or more data portions from the secondary cache to physical storage, said flushing processing using a first flushing threshold specified for the first pool of logical devices to determine whether a third data portion of the first pool of logical devices is flushed from the secondary cache, wherein the first flushing threshold denotes a maximum number of secondary cache pages used for storing data of each logical device of the first pool and the flushing processing applies the first flushing threshold individually to each logical device of the first pool, said flushing processing using a second flushing threshold specified for the second pool of logical devices to determine whether a fourth data portion of the second pool of logical devices is flushed from the secondary cache, wherein the second flushing threshold denotes a maximum number of secondary cache pages used for storing data of each logical device of the second pool and the flushing processing applies the second flushing threshold individually to each logical device of the second pool.
16. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of caching data comprising:
performing promotion processing to promote one or more data portions to a secondary cache, wherein said secondary cache and a primary data cache are included in a plurality of caching layers, said promotion processing using a first promotion threshold specified for a first pool of logical devices to determine whether a first data portion of the first pool of logical devices is promoted to the secondary cache, said promotion processing using a second promotion threshold specified for a second pool of logical devices to determine whether a second data portion of the second pool of logical devices is promoted to the secondary cache;
determining whether a current number of free cache pages in the secondary cache is greater than a specified minimum number of secondary cache pages; and
responsive to determining the current number of free cache pages in the secondary cache is not greater than the specified minimum number of free pages, performing flushing processing to flush one or more data portions from the secondary cache to physical storage, said flushing processing using a first flushing threshold specified for the first pool of logical devices to determine whether a third data portion of the first pool of logical devices is flushed from the secondary cache, wherein the first flushing threshold denotes a maximum number of secondary cache pages used for storing data of each logical device of the first pool and the flushing processing applies the first flushing threshold individually to each logical device of the first pool, said flushing processing using a second flushing threshold specified for the second pool of logical devices to determine whether a fourth data portion of the second pool of logical devices is flushed from the secondary cache, wherein the second flushing threshold denotes a maximum number of secondary cache pages used for storing data of each logical device of the second pool and the flushing processing applies the second flushing threshold individually to each logical device of the second pool.
17. The non-transitory computer readable medium of claim 16 , wherein the promotion processing includes:
determining whether a first count, denoting a number of I/Os referencing the first data portion, is equal to or greater than the first promotion threshold;
responsive to determining the first count is equal to or greater than the first promotion threshold, promoting the first data portion to the secondary cache;
determining whether a second count, denoting a number of I/Os referencing the second data portion, is equal to or greater than the second promotion threshold; and
responsive to determining the second count is equal to or greater than the second promotion threshold, promoting the second data portion to the secondary cache.
18. The non-transitory computer readable medium of claim 16 , wherein the third data portion is stored at a first logical address of a first logical device, said first logical device being included in the first pool, and wherein the flushing processing includes:
determining whether a third count, denoting a number of cache pages of the secondary cache include cached data of the first logical device, exceeds the first flushing threshold; and
responsive to determining the third count exceeds the first flushing threshold, flushing the third data portion from the secondary cache to physical storage.
19. The non-transitory computer readable medium of claim 18 , wherein data portions of the first logical device are flushed from the secondary cache to physical storage until the third count is below the first flushing threshold.
20. The non-transitory computer readable medium of claim 19 , wherein the fourth data portion is stored at a second logical address of a second logical device, said second logical device being included in the second pool, and wherein the flushing processing includes:
determining whether a fourth count, denoting a number of cache pages of the secondary cache include cached data of the second logical device, exceeds the second flushing threshold; and
responsive to determining the fourth count exceeds the second flushing threshold, flushing the fourth data portion from the secondary cache to physical storage.
21. The non-transitory computer readable medium of claim 18 , wherein data portions of the second logical device are flushed from the secondary cache to physical storage until the fourth count is below the second flushing threshold.
22. The non-transitory computer readable medium of claim 16 , wherein a first set of values are specified for the first pool, said first set of values including a first priority for the first pool, the first flushing threshold for the first pool and the first promotion threshold for the first pool, the first flushing threshold and the first promotion threshold being determined in accordance with the first priority; and wherein a second set of values are specified for the second pool, said second set of values including a second priority for the second pool, the second flushing threshold for the second pool and the second promotion threshold for the second pool, the second flushing threshold and the second promotion threshold being determined in accordance with the second priority.