Techniques for space reservation in a storage environment
Improved techniques and systems are disclosed for ensuring that physical storage is available for cached allocating writes in a thinly provisioned storage environment. By monitoring the size of the cached allocating writes in the file system cache and taking cache occupancy reduction actions when criteria for cache reduction are fulfilled, caching of allocating writes that do not have a corresponding physical storage allocation can be eliminated or reduced to a user-configurable maximum without disabling caching of allocating writes. Using these techniques, allocating writes may be cached without risk of data loss.
1. A method comprising:
caching allocating writes on a cache storage, wherein the cached allocating writes are writes not having a previously allocated physical storage space that are later to be stored on a physical storage separate from the cache storage that stores the cached allocating writes;
in response to caching allocating writes, reserving a block of the physical storage separate from the cache storage that stores the cached allocating writes;
comparing a total cumulative size of the cached allocating writes to an upper threshold; and
when the total cumulative size of the cached allocating writes exceeds the upper threshold, taking a cache occupancy reduction action;
wherein the reservation of the block of the physical storage, the caching allocating writes, and the cache reduction action are managed to avoid failures to commit to write to the physical storage.
2. The method of claim 1 , wherein the cache occupancy reduction action is an action selected from the group of: performing background flushing to physical storage of the cached allocating writes and reserving additional physical storage.
3. The method of claim 1 , further comprising: continuing to cache allocating writes;
comparing an unbacked data amount to an unbacked data threshold; and
when the unbacked data amount exceeds the unbacked data threshold, taking one or more additional cache occupancy reduction actions.
4. The method of claim 3 , wherein the unbacked data amount is the total size of the reserved physical storage subtracted from the total cumulative size of the cached allocating writes.
5. The method of claim 3 , wherein caching allocating writes is paused when the unbacked data amount exceeds the unbacked data threshold.
6. The method of claim 3 , wherein the unbacked data threshold is a percentage of the size of the reserved physical storage.
7. The method of claim 1 , wherein reserving a block of physical storage comprises making an API call to a reserved space management agent.
8. The method of claim 1 , wherein the upper threshold is a percentage of the size of the reserved physical storage.
9. The method of claim 1 , further comprising pausing caching allocating writes until the cache occupancy reduction action is completed.
10. The method of claim 1 , wherein the cache occupancy reduction action comprises performing background flushing to physical storage of the cached allocating writes and wherein the cached allocating writes are written to physical storage other than the reserved physical storage.
11. The method of claim 1 , wherein the cache occupancy reduction action comprises performing background flushing to physical storage of the cached allocating writes and wherein taking the cache occupancy reduction action has started, further comprising:
comparing the total cumulative size of the cached allocating writes to a lower threshold; and
stopping the background flushing to physical storage of the cached allocating writes if the total cumulative size of the cached allocating writes is smaller than the lower threshold.
12. A system comprising:
a program memory;
at least one processor communicatively coupled to the program memory and a storage device and configured to execute program instructions to cause the at least one processor to:
caching allocating writes on a cache storage, wherein the cached allocating writes are writes not having a previously allocated physical storage space that are later to be stored to physical storage in the storage device storage separate from the cache storage that stores the cached allocating writes;
in response to caching allocating writes, reserve a block of the physical storage separate from the cache storage that stores the cached all ocating writes;
compare a total cumulative size of the cached allocating writes to an upper threshold; and
when the total cumulative size of the cached allocating writes exceeds the upper threshold, take a cache occupancy reduction action;
wherein the reservation of the block of the physical storage, the caching allocating writes, and the cache reduction action are managed to avoid failures to commit to write to the physical storage.
13. The system of claim 12 , wherein the cache occupancy reduction action is an action selected from the group of: performing background flushing to physical storage of the cached allocating writes and reserving additional physical storage.
14. The system of claim 12 , wherein the program instructions further cause the processor to:
continue to cache allocating writes;
compare an unbacked data amount to an unbacked data threshold; and
when the unbacked data amount exceeds the unbacked data threshold, take one or more additional cache occupancy reduction actions.
15. The system of claim 12 , wherein the cache occupancy reduction action comprises performing background flushing to physical storage of the cached all ocating writes, wherein, the program instructions further cause the processor to:
when the cache occupancy reduction action has started:
compare the total cumulative size of the cached allocating writes to a lower threshold; and
stop the background flushing to physical storage of the cached allocating writes if the total cumulative size of the cached allocating writes is smaller than the lower threshold.
16. The system of claim 12 , wherein the all ocating writes are cached in a cache memory coupled to the at least one processor.
17. The system of claim 12 , further comprising a file system executed by the at least one processor and configured to issue allocating writes.
18. The system of claim 12 , wherein the program instructions further cause the at least one processor to reserve the block of storage in the storage device by issuing a storage reservation API command to a reserve space management agent in the storage device.
19. The system of claim 12 , wherein the program instructions further cause the at least one processor to execute a cache management agent, wherein the comparison is performed by the cache management agent.
20. The system of claim 19 , wherein the program instructions further cause the at least one processor to take the cache occupancy reduction action by causing the cache management agent to trigger a cache occupancy reduction agent to perform the cache occupancy reduction action.