Distributed storage resource scheduler and load balancer
Distributed storage resources are managed based on data collected from online monitoring of workloads on the storage resources and performance characteristics of the storage resources. Load metrics are calculated from the collected data and used to identify workloads that are migration candidates and storage units that are migration destinations, so that load balancing across the distributed storage resources can be achieved.
1. A method of managing distributed storage resources including a first storage unit and a second storage unit, comprising:
while the first storage unit and the second storage unit are online, monitoring workloads on the first storage unit and the second storage unit, the workloads being associated with objects stored in the first storage unit and the second storage unit, and monitoring performance of the first storage unit and the second storage unit, wherein the performance of the first storage unit is dependent on the number of outstanding input-output requests to the first storage unit and average latency of input-output requests to the first storage unit, and wherein the performance of the second storage unit is dependent on the number of outstanding input-output requests to the second storage unit and average latency of input-output requests to the second storage unit;
calculating a load metric for each of the first and second storage units based on measurements of the monitored workloads on the respective storage unit and a measurement of the monitored performance of the respective storage unit;
selecting an object to be migrated between the first storage unit and the second storage unit, wherein the object to be migrated is selected from the storage unit with the higher calculated load metric and comprises a disk image of a virtual machine; and
migrating the disk image of the virtual machine between the first storage unit and the second storage unit without substantially interrupting the virtual machine.
2. The method according to claim 1 , wherein a workload associated with an object is proportional to the number of outstanding input-output requests to the object and an average size of the input-output requests to the object.
3. The method according to claim 2 , wherein the workload associated with the object is further proportional to the percentage of reads in the input-output requests to the object and a value representative of the randomness of the input-output requests to the object.
4. The method according to claim 1 , wherein the first storage unit is a first logical unit number exposed by a first storage array and the second storage unit is a second logical unit number exposed by a second storage array.
5. The method according to claim 4 , wherein the first storage array and the second storage array are physically separate storage arrays.
6. A non-transitory computer readable storage medium comprising instructions which, when executed in a computing device for monitoring a first storage unit and a second storage unit, causes the computing device to carry out the steps of:
while the first storage unit and the second storage unit are online, monitoring workloads on the first storage unit and the second storage unit, the workloads being associated with objects stored in the first storage unit and the second storage unit, and monitoring performance of the first storage unit and the second storage unit, wherein the performance of the first storage unit is dependent on the number of outstanding input-output requests to the first storage unit and average latency of input-output requests to the first storage unit, and wherein the performance of the second storage unit is dependent on the number of outstanding input-output requests to the second storage unit and average latency of input-output requests to the second storage unit;
calculating a load metric for each of the first and second storage units based on measurements of the monitored workloads on the respective storage unit and a measurement of the monitored performance of the respective storage unit;
selecting an object to be migrated between the first storage unit and the second storage unit, wherein the object to be migrated is selected from the storage unit with the higher calculated load metric and comprises a disk image of a virtual machine; and
migrating the disk image of the virtual machine between the first storage unit and the second storage unit without substantially interrupting the virtual machine.
7. The non-transitory computer readable storage medium according to claim 6 , wherein a workload associated with an object is proportional to the number of outstanding input-output requests to the object and an average size of the input-output requests to the object.
8. The non-transitory computer readable storage medium according to claim 7 , wherein the workload associated with the object is further proportional to the percentage of reads in the input-output requests to the object and a value representative of the randomness of the input-output requests to the object.
9. A method of performing load balancing across storage units having different vendor configurations based on online monitoring of workloads and input-output requests to the storage units, comprising:
monitoring workloads on a first storage unit and on a second storage unit, each workload being associated with an object stored in the first or the second storage unit;
monitoring an online performance of the first storage unit, wherein the online performance of the first storage unit is dependent on the number of outstanding input-output requests to the first storage unit and average latency of input-output requests to the first storage unit;
monitoring an online performance of a second storage unit, wherein the online performance of the second storage unit is dependent on the number of outstanding input-output requests to the second storage unit and average latency of input-output requests to the second storage unit;
calculating a load metric for each of the first and second storage units based on measurements of the monitored workloads on the respective storage unit and a measurement of the monitored performance of the respective storage unit;
selecting an object for migration from the storage unit with the higher calculated load metric, wherein the selected object comprises a disk image of a virtual machine; and
migrating the disk image between the first storage unit and the second storage unit without substantially interrupting the virtual machine.
10. The method according to claim 9 , wherein a workload associated with an object is proportional to the number of outstanding input-output requests to the object and an average size of the input-output requests to the object.
11. The method according to claim 10 , wherein the workload associated with the object is further proportional to the percentage of reads in the input-output requests to the object and a value representative of the randomness of the input-output requests.
12. The method according to claim 1 , wherein the measurement of the monitored performance of the first storage unit is determined based on a linear fit which minimizes an error for a set of data points comprising the number of outstanding input-output requests to the first storage unit and the average latency of input-output requests to the first storage unit over respective time intervals, and wherein the measurement of the monitored performance of the second storage unit is determined based on a linear fit which minimizes an error for a set of data points comprising the number of outstanding input-output requests to the second storage unit and the average latency of input-output requests to the second storage unit over respective time intervals.
13. The non-transitory computer readable storage medium according to claim 6 , wherein the measurement of the monitored performance of the first storage unit is determined based on a linear fit which minimizes an error for a set of data points comprising the number of outstanding input-output requests to the first storage unit and the average latency of input-output requests to the first storage unit over respective time intervals, and wherein the measurement of the monitored performance of the second storage unit is determined based on a linear fit which minimizes an error for a set of data points comprising the number of outstanding input-output requests to the second storage unit and the average latency of input-output requests to the second storage unit over respective time intervals.
14. The method according to claim 9 , wherein the measurement of the monitored performance of the first storage unit is determined based on a linear fit which minimizes an error for a set of data points comprising the number of outstanding input-output requests to the first storage unit and the average latency of input-output requests to the first storage unit over respective time intervals, and wherein the measurement of the monitored performance of the second storage unit is determined based on a linear fit which minimizes an error for a set of data points comprising the number of outstanding input-output requests to the second storage unit and the average latency of input-output requests to the second storage unit over respective time intervals.