Management of unmap processing rates in distributed and shared data storage volumes
The disclosure herein describes managing a rate of processing unmap requests for a data storage volume. Unmap requests are received from a cluster of active hosts that are associated with the data storage volume. Latency data values of each active host are then accessed. A long-term cluster latency average value is calculated based on the accessed latency data values of all active hosts over a long-term time period and a short-term cluster latency average value is calculated based on the accessed latency data values of all active hosts over a short-term time period. An unmap rate adjustment value is calculated based on a difference between the long-term cluster latency average value and the short-term cluster latency average value. The rate of processing unmap requests for the data storage volume is adjusted based on the unmap rate adjustment value and the unmap requests are performed based on the adjusted rate.
1. A computerized method for managing a rate of processing unmap requests for a data storage volume, the method comprising:
receiving and buffering, by a processor, unmap requests from a cluster of active hosts that are associated with the data storage volume;
calculating, by the processor, an unmap rate adjustment value based on a cluster latency difference between a first cluster latency average value of the cluster of active hosts based on a first time period and a second cluster latency average value of the cluster of active hosts based on a second time period;
adjusting, by the processor, the rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value; and
performing, by the processor, the buffered unmap requests based on the adjusted rate of processing unmap requests of the data storage volume.
2. The computerized method of claim 1 , further comprising:
calculating, by the processor, the first cluster latency average value as a long-term cluster latency average value based on latency data values of all active hosts in the cluster over a long-term time period; and
calculating, by the processor, the second cluster latency average value as a short-term cluster latency average value based on the latency data values of all active hosts in the cluster over a short-term time period, wherein the long-term time period is longer than the short-term time period.
3. The computerized method of claim 2 , further comprising:
calculating, for each active host of the cluster, a long-term host latency average value based on the latency data values of the active host over the long-term time period; and
calculating, for each active host of the cluster, a short-term host latency average value based on the latency data values for the active host over the short-term time period;
wherein calculating the unmap rate adjustment value includes calculating a host-specific unmap rate adjustment value for each active host of the cluster based on combining the cluster latency difference and a host latency difference between the long-term host latency average value of the active host and the short-term host latency average value of the active host; and
wherein adjusting the rate of processing unmap requests includes adjusting, for each active host of the cluster, a host-specific rate of processing unmap requests based on the calculated host-specific unmap rate adjustment value for the active host, such that buffered unmap requests associated with the active host are performed based on the adjusted host-specific rate of processing unmap requests.
4. The computerized method of claim 3 , wherein combining the cluster latency difference and the host latency difference includes applying a weight value to the cluster latency difference or the host latency difference or both.
5. The computerized method of claim 1 , wherein the first and second cluster latency average values are based on latency data associated with a subset of operations out of a plurality of operations associated with the data storage volume, wherein the subset of operations are higher priority operations than other operations in the plurality of operations.
6. The computerized method of claim 1 , wherein adjusting the rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value is further based on the unmap rate adjustment value exceeding a defined threshold.
7. The computerized method of claim 1 , wherein adjusting the rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value includes applying a rate adjustment factor to the unmap rate adjustment value based on a latency pattern that is defined based on at least one of the following: a current time and a current date.
8. A computer system comprising:
a processor at a device associated with a data storage volume;
a non-transitory computer readable medium having stored thereon program code for managing a rate of processing unmap requests for the data storage volume, the program code causing the processor to:
receive and buffer unmap requests from a cluster of active hosts that are associated with the data storage volume;
calculate an unmap rate adjustment value based on a cluster latency difference between a first cluster latency average value of the cluster of active hosts based on a first time period and a second cluster latency average value of the cluster of active hosts based on a second time period;
adjust the rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value; and
perform the buffered unmap requests based on the adjusted rate of processing unmap requests of the data storage volume.
9. The computing system of claim 8 , the program code further causing processor to:
calculate the first cluster latency average value as a long-term cluster latency average value based on latency data values of all active hosts in the cluster over a long-term time period; and
calculate the second cluster latency average value as a short-term cluster latency average value based on the latency data values of all active hosts in the cluster over a short-term time period, wherein the long-term time period is longer than the short-term time period.
10. The computing system of claim 9 , the program code further causing processor to:
calculate a long-term host latency average value based on the latency data values of the active host over the long-term time period; and
calculate a short-term host latency average value based on the latency data values for the active host over the short-term time period;
wherein calculating the unmap rate adjustment value includes calculating a host-specific unmap rate adjustment value for each active host of the cluster based on combining the cluster latency difference and a host latency difference between the long-term host latency average value of the active host and the short-term host latency average value of the active host; and
wherein adjusting the rate of processing unmap requests includes adjusting, for each active host of the cluster, a host-specific rate of processing unmap requests based on the calculated host-specific unmap rate adjustment value for the active host, such that buffered unmap requests associated with the active host are performed based on the adjusted host-specific rate of processing unmap requests.
11. The computing system of claim 10 , wherein combining the cluster latency difference and the host latency difference includes applying a weight value to the cluster latency difference or the host latency difference or both.
12. The computing system of claim 8 , wherein the first and second cluster latency average values are based on latency data associated with a subset of operations out of a plurality of operations associated with the data storage volume, wherein the subset of operations are higher priority operations than other operations in the plurality of operations.
13. The computing system of claim 8 , wherein adjusting the rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value is further based on the unmap rate adjustment value exceeding a defined threshold.
14. The computing system of claim 8 , wherein adjusting the rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value includes applying a rate adjustment factor to the unmap rate adjustment value based on a latency pattern that is defined based on at least one of the following: a current time and a current date.
15. A non-transitory computer readable storage medium having stored thereon program code executable by a computer system associated with a data storage volume, the program code embodying a method comprising:
receiving and buffering, by a processor, unmap requests from a cluster of active hosts that are associated with the data storage volume;
calculating, by the processor, an unmap rate adjustment value based on a cluster latency difference between a first cluster latency average value of the cluster of active hosts based on a first time period and a second cluster latency average value of the cluster of active hosts based on a second time period;
adjusting, by the processor, a rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value; and
performing, by the processor, the buffered unmap requests based on the adjusted rate of processing unmap requests of the data storage volume.
16. The non-transitory computer readable storage medium of claim 15 , the program code further embodying a method comprising:
calculating, by the processor, the first cluster latency average value as a long-term cluster latency average value based on latency data values of all active hosts in the cluster over a long-term time period; and
calculating, by the processor, the second cluster latency average value as a short-term cluster latency average value based on the latency data values of all active hosts in the cluster over a short-term time period, wherein the long-term time period is longer than the short-term time period.
17. The non-transitory computer readable storage medium of claim 16 , the program code further embodying a method comprising:
calculating, for each active host of the cluster, a long-term host latency average value based on the latency data values of the active host over the long-term time period; and
calculating, for each active host of the cluster, a short-term host latency average value based on the latency data values for the active host over the short-term time period;
wherein calculating the unmap rate adjustment value includes calculating a host-specific unmap rate adjustment value for each active host of the cluster based on combining the cluster latency difference and a host latency difference between the long-term host latency average value of the active host and the short-term host latency average value of the active host; and
wherein adjusting the rate of processing unmap requests includes adjusting, for each active host of the cluster, a host-specific rate of processing unmap requests based on the calculated host-specific unmap rate adjustment value for the active host, such that buffered unmap requests associated with the active host are performed based on the adjusted host-specific rate of processing unmap requests.
18. The non-transitory computer readable storage medium of claim 15 , wherein the first and second cluster latency average values are based on latency data associated with a subset of operations out of a plurality of operations associated with the data storage volume, wherein the subset of operations are higher priority operations than other operations in the plurality of operations.
19. The non-transitory computer readable storage medium of claim 15 , wherein adjusting the rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value is further based on the unmap rate adjustment value exceeding a defined threshold.
20. The non-transitory computer readable storage medium of claim 15 , wherein adjusting the rate of processing unmap requests for the data storage volume based on the calculated unmap rate adjustment value includes applying a rate adjustment factor to the unmap rate adjustment value based on a latency pattern that is defined based on at least one of the following: a current time and a current date.