Lowering emergency capacity reservations for storage systems
Aspects of the disclosure are directed to reducing or eliminating emergency capacity usage in a storage system by including an in-process space reclamation and/or one or more mitigation mechanisms. The in-process space reclamation can be implemented in addition to a background space reclamation. The one or more mitigation mechanisms can be associated with a basis for write requests. The in-process space reclamation and/or mitigation mechanisms allow for lowering emergency capacity reservation, thus saving storage costs.
1 . A method for reducing emergency storage capacity of a storage system, the method comprising:
monitoring, by one or more processors, an amount of time since a background space reclamation has been performed;
determining, by the one or more processors, that the amount of time since the background space reclamation has been performed meets or exceeds an idling threshold;
based on determining that the amount of time meets or exceeds the idling threshold, triggering, by the one or more processors, an in-process space reclamation separate from the background space reclamation, the in-process space reclamation being bound to one or more write processes for the storage system by running on the same hardware and being programmed in a same memory address space as the one or more write processes;
monitoring, by the one or more processors, a storage usage for the storage system;
determining, by the one or more processors, that the storage usage meets or exceeds a storage usage threshold; and
based on determining that the storage usage meets or exceeds the storage usage threshold, triggering, by the one or more processors, the in-process space reclamation and adjusting, by the one or more processors, a basis for the write processes, wherein the adjusting further comprises at least one of:
transcoding data to a wider encoding scheme; or
performing data compression on incoming traffic, including initiating compression of incoming traffic or increasing a compression ratio for the incoming traffic.
2 . The method of claim 1 , wherein the in-process space reclamation is bound to the one or more write processes by having the same software package as the one or more write processes.
3 . The method of claim 1 , further comprising:
monitoring, by the one or more processors, a file density for the storage system, the file density being based on a number of blocks overwritten in the storage system;
determining, by the one or more processors, that the file density meets or is below a density threshold; and
triggering, by the one or more processors, the in-process space reclamation.
4 . The method of claim 1 , wherein adjusting the basis for the write processes further comprises routing incoming traffic or a portion of the incoming traffic to a hard disk drive (HDD) if the storage system is a solid state drive (SSD) or to a SSD if the storage system is a HDD.
5 . The method of claim 1 , wherein adjusting the basis for the write processes further comprises moving data to at least one of a wider redundant array of independent disks (RAID), remote storage, or storage at a different geographical location.
6 . The method of claim 1 , wherein the storage usage threshold comprises multiple threshold levels and different mitigation mechanisms are performed at each of the multiple threshold levels.
7 . The method of claim 1 , further comprising triggering, by the one or more processors, a space reclamation that runs on older software than the in-process space reclamation.
8 . A system comprising:
one or more processors; and
one or more storage devices coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for reducing emergency storage capacity of a storage system, the operations comprising:
monitoring an amount of time since a background space reclamation has been performed;
determining that the amount of time since the background space reclamation has been performed meets or exceeds an idling threshold;
based on determining that the amount of time meets of exceeds the idling threshold, triggering an in-process space reclamation separate from the background space reclamation, the in-process space reclamation being bound to one or more write processes for the storage system by running on the same hardware and being programmed in a same memory address space as the one or more write processes;
monitoring a storage usage for the storage system;
determining that the storage usage meets or exceeds a storage usage threshold; and
based on determining that the storage usage meets or exceeds the storage usage threshold, triggering the in-process space reclamation and adjusting a basis for the write processes, wherein the adjusting further comprises at least one of:
transcoding data to a wider encoding scheme; or
performing data compression on incoming traffic, including initiating compression of incoming traffic or increasing a compression ratio for the incoming traffic.
9 . The system of claim 8 , wherein the in-process space reclamation is bound to the one or more write processes by having the same software package as the one or more write processes.
10 . The system of claim 8 , wherein the operations further comprise:
monitoring a file density for the storage system, the file density being based on a number of blocks overwritten in the storage system;
determining that the file density meets or is below a density threshold; and
triggering the in-process space reclamation.
11 . A non-transitory computer readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for reducing emergency storage capacity of a storage system, the operations comprising:
monitoring an amount of time since a background space reclamation has been performed;
determining that the amount of time since the background space reclamation has been performed meets or exceeds an idling threshold;
based on determining that the amount of time meets or exceeds the idling threshold, triggering an in-process space reclamation separate from the background space reclamation, the in-process space reclamation being bound to one or more write processes for the storage system by running on the same hardware and being programmed in a same memory address space as the one or more write processes;
monitoring a storage usage for the storage system;
determining that the storage usage meets or exceeds a storage usage threshold; and
based on determining that the storage usage meets or exceeds the storage usage threshold, triggering the in-process space reclamation and adjusting a basis for the write processes, wherein the adjusting further comprises at least one of:
transcoding data to a wider encoding scheme; or
performing data compression on incoming traffic, including initiating compression of incoming traffic or increasing a compression ratio for the incoming traffic.
12 . The non-transitory computer readable medium of claim 11 , wherein the operations further comprise:
monitoring a file density for the storage system, the file density being based on a number of blocks overwritten in the storage system;
determining that the file density meets or is below a density threshold; and
triggering the in-process space reclamation.