Performance evaluation and comparison of storage systems
Described embodiments provide storage system evaluation and comparison processes. An aspect includes sampling data points for a workload running on system over a sampling period. The data points indicate a performance metric with respect to operational characteristics of the system. An aspect further includes subtracting a system specification value from each of the averaged sampled data points, thereby producing deviation values reflecting a deviation of the sampled data points from the system specification value. An aspect also includes averaging the sampled data points, calculating a standard deviation of the averaged sampled data points, and dividing the variance value by the standard deviation, thereby producing a modified performance value that accounts for a deviation in the operational characteristics of the system over the sampling period.
1. A method comprising:
sampling data points for a workload running on a system over a sampling period, the data points indicating a performance metric with respect to operational characteristics of the system;
subtracting a system specification value from each of the sampled data points, the subtracting producing deviation values reflecting a deviation of each of the sampled data points from the system specification value;
averaging the deviation values with respect to the sampled data points;
calculating a standard deviation of the sampled data points;
dividing the averaged deviation values by the standard deviation, the dividing producing a modified performance value that accounts for a deviation in the operational characteristics of the system over the sampling period;
comparing the modified performance value to a benchmark performance value; and
implementing an action to be taken with respect to the system based on results of the comparing.
2. The method of claim 1 , wherein the performance metric is operating bandwidth and the workload is synthetic.
3. The method of claim 1 , wherein the performance metric is a function of latency and bandwidth.
4. The method of claim 1 , wherein the system specification value is a system-defined value representing minimum required bandwidth for an average workload.
5. The method of claim 1 , wherein the system specification value is a system-defined value representing minimum required bandwidth for a peak workload.
6. The method of claim 1 , further comprising:
sampling data points for a second workload, each of the data points indicating a performance metric with respect to operational characteristics;
for the sampled data points of the second workload:
subtracting a system specification value from each of the sampled data points, the subtracting producing deviation values reflecting a deviation of each of the sampled data points from the system specification value;
averaging the deviation values with respect to the sampled data points;
calculating a standard deviation of the sampled data points;
dividing the averaged deviation values by the standard deviation, the dividing producing a modified performance value that accounts for a deviation in the operational characteristics of the system over the sampling period; and
comparing the modified performance value of the first workload with the modified performance value of the second workload.
7. The method of claim 6 , wherein the second workload is one of:
a similarly configured workload running subsequent to the workload on the system;
a workload running on the system subsequent to updates made to system code levels between execution of the first workload and the second workload; and
a workload operating on a second system.
8. A system comprising:
a memory comprising computer-executable instructions; and
a processor executing the computer-executable instructions, the computer-executable instructions when executed by the processor cause the processor to perform operations comprising:
sampling data points for a workload running on a system over a sampling period, the data points indicating a performance metric with respect to operational characteristics of the system;
subtracting a system specification value from each of the sampled data points, the subtracting producing deviation values reflecting a deviation of each of the sampled data points from the system specification value;
calculating a standard deviation of the sampled data points;
dividing the averaged deviation values by the standard deviation, the dividing producing a modified performance value that accounts for a deviation in the operational characteristics of the system over the sampling period;
comparing the modified performance value to a benchmark performance value; and
implementing an action to be taken with respect to the system based on results of the comparing.
9. The system of claim 8 , wherein the performance metric is operating bandwidth and the workload is synthetic.
10. The system, of claim 8 , wherein the performance metric is a function of latency and bandwidth.
11. The system of claim 8 , wherein the system specification value is a system-defined value representing minimum required bandwidth for an average workload.
12. The system of claim 8 , wherein the system specification value is a system-defined value representing minimum required bandwidth for a peak workload.
13. The system of claim 8 , wherein the operations further comprise:
sampling data points for a second workload, each of the data points indicating a performance metric with respect to operational characteristics;
for the sampled data points of the second workload:
subtracting a system specification value from each of the sampled data points, the subtracting producing deviation values reflecting a deviation of each of the sampled data points from the system specification value;
averaging the deviation values with respect to the sampled data points;
calculating a standard deviation of the sampled data points;
dividing the averaged deviation values by the standard deviation, the dividing producing a modified performance value that accounts for a deviation in the operational characteristics of the system over the sampling period; and
comparing the modified performance value of the first workload with the modified performance value of the second workload.
14. The system of claim 13 , wherein the second workload is one of:
a similarly configured workload running subsequent to the workload on the system;
a workload running on the system subsequent to updates made to system code levels between execution of the first workload and the second workload; and
a workload operating on a second system.
15. A computer program product embodied on a non-transitory computer readable medium, the computer program product including instructions that, when executed by a computer, causes the computer to perform operations comprising:
sampling data points for a workload running on a system over a sampling period, the data points indicating a performance metric with respect to operational characteristics of the system;
subtracting a system specification value from each of the sampled data points, the subtracting producing deviation values reflecting a deviation of each of the sampled data points from the system specification value;
averaging the deviation values with respect to the sampled data points;
calculating a standard deviation of the sampled data points;
dividing the averaged deviation values by the standard deviation, the dividing producing a modified performance value that accounts for a deviation in the operational characteristics of the system over the sampling period;
comparing the modified performance value to a benchmark performance value; and
implementing an action to be taken with respect to the system based on results of the comparing.
16. The computer program product of claim 15 , wherein the performance metric is at least one of:
operating bandwidth and the workload is synthetic; and
a function of latency and bandwidth.
17. The computer program product of claim 15 , wherein the system specification value is a system-defined value representing minimum required bandwidth for an average workload.
18. The computer program product of claim 15 , wherein the system specification value is a system-defined value representing minimum required bandwidth for a peak workload.
19. The computer program product of claim 15 , wherein the operations further comprise:
sampling data points for a second workload, each of the data points indicating a performance metric with respect to operational characteristics;
for the sampled data points of the second workload:
subtracting a system specification value from each of the sampled data points, the subtracting producing deviation values reflecting a deviation of each of the sampled data points from the system specification value;
calculating a standard deviation of the sampled data points;
dividing the averaged deviation values by the standard deviation, the dividing producing a modified performance value that accounts for a deviation in the operational characteristics of the system over the sampling period; and
comparing the modified performance value of the first workload with the modified performance value of the second workload.
20. The computer program product of claim 19 , wherein the second workload is one of:
a similarly configured workload running subsequent to the workload on the system;
a workload running on the system subsequent to updates made to system code levels between execution of the first workload and the second workload; and
a workload operating on a second system.