System and method for statistical performance monitoring
View Patent ↗A method using statistical parameters (e.g. mean, standard deviation, exceptional values) of performance monitoring metrics to substantially reduce the quantity of performance monitoring data collected and reported, make system performance monitoring scalable and enhance the readability of the system performance display. The number of metrics monitored may be reduced by monitoring only one of any two metrics that are closely correlated.
1. A computer-implemented method of reducing reporting of system metrics, the method comprising:
obtaining first sampled values of a first system metric with at least one agent;
determining a weighted moving average of a plurality of the first sampled values;
deriving a non-reporting range from the weighted moving average, the non-reporting range defined by at least one threshold parameter determined from the weighted moving average; and
withholding reporting of one or more of the first sampled values to a system performance monitor only if the one or more first sampled values are within the non-reporting range,
wherein the weighted moving average includes an exponential weighted moving average d n (w) for an n'th sample of the first system metric, and wherein the exponential weighted moving average d n (w) is determined from an exponential weighted moving average d n−1 (w) for an (n−1)'th sample and a weight w, where d n (w)=d n w+ d n−1 (1−w).
2. The method of claim 1 , further comprising reporting one or more of the first sampled values to the system performance monitor only if the one or more first sampled values are outside the non-reporting range.
3. The method of claim 2 , further comprising;
receiving, at the system performance monitor, the one or more first sampled values reported; and
assuming, at the system performance monitor, an average for the one or more first sampled values not reported.
4. The method of claim 1 , wherein the at least one threshold parameter is determined by a standard deviation of the weighted moving average.
5. The method of claim 1 , wherein the at least one threshold parameter is determined by a percentage of the weighted moving average.
6. The method of claim 1 , wherein the weight w equals c/N, wherein c is a real number for a weighting factor, and N is an integer for a window size of samples.
7. The method of claim 6 , wherein the weight w is between 1/N and 2/N.
8. The method of claim 6 , wherein the window size N is determined by a confidence interval cl, the tolerable variance error e v: , wherein
e
v
=
100
f
(
cl
)
2
N
,
wherein f(cl) is the (1+cl/100)/2-quantile of the unit normal distribution.
9. The method of claim 6 , wherein the weighting factor c equals 1, and wherein the window size N equals n for the n'th sample of the first system metric, whereby the exponential weighted moving average d n (w)is a running average d n for the n'th sample of the first system metric.
10. The method of claim 9 , wherein the act of deriving the non-reporting range comprises:
calculating a first threshold parameter for the non-reporting range to be ( d n −aσ n ), and
calculating a second threshold parameter for the non-reporting range to be ( d n+bσ n ),
wherein a and b are two constant real numbers,
wherein σ n is a standard deviation for the n'th sample of the first system metric, and
wherein the non-reporting range is defined between the first and second parameters.
11. The method of claim 9 , wherein the running average d n is incremented using a running average d n−1 for the (n−1)'th sample of the first system metric and a first sample value d n for the n'th sample by computing
d
_
n
=
(
n
-
1
)
d
_
n
-
1
+
d
n
n
.
12. The method of claim 9 , wherein the standard deviation σ n for the running average d n is determined from a current variance σ n 2 calculated by σ n 2 +S n /n, wherein S n =S n−1 +(n−1)(d n − d n−1 ) 2 /n, and wherein
S
n
=
∑
i
=
1
n
(
d
i
-
d
_
n
)
2
.
13. The method of claim 1 , wherein the act of withholding reporting of the one or more first sampled values to the system performance monitor only if the one or more of the first sampled values are within the non-reporting range further comprises:
calculating a running average d i of a plurality of the first sample values; and
reporting the running average d i to the system performance monitor.
14. The method of claim 13 , further comprising:
calculating a new running average d j of a plurality of the first sample values; and
reporting the new running average d j instead of the former running average d i to the system performance monitor if | d i − d j | is greater than a replacement threshold.
15. The method of claim 13 , wherein the replacement threshold is a standard deviation σ n for the new running average d i .
16. The method of claim 1 , further comprising:
obtaining second sampled values of a second system metric with at least one agent;
calculating a correlation coefficient cc between a plurality of the first and second sampled values;
determining an absolute value of the correlation coefficient cc; and
withholding reporting of the plurality of second sample values to the system performance monitor if the absolute value |cc| is greater than a threshold.
17. The method of claim 16 , further comprising discontinuing the act of obtaining second sampled values of the second system metric if the absolute value |cc| is greater than the threshold.
18. The method of claim 16 , further comprising:
reporting one or more of the second sampled values to the system performance monitor if the absolute value |cc| is less than the threshold; and
continuing the act of obtaining second sampled values of the second system metric with at least one agent.
19. A program storage device, readable by a programmable control device, comprising instructions stored on the program stage device for causing the programmable control device to perform a method according to any one of claims 1 through 18 .
20. A computer-implemented method of reducing reporting of system metrics, comprising:
obtaining first sampled values of a first system metric with at least one agent;
reporting one or more of the first sampled values to a system performance monitor;
obtaining second sampled values of a second system metric with at least one agent;
calculating a correlation coefficient cc between a plurality of the first and second sampled values;
determining an absolute value of the correlation coefficient cc; and
withholding reporting of the plurality of second sampled values to the system performance monitor if the absolute value |cc| is greater than a threshold,
wherein the obtaining the first sampled values includes deriving a non-reporting range defined by at least one threshold parameter determined from a weighted moving average of the first sampled values, and the reporting one or more of the first sampled values to a system performance monitor includes withholding reporting of one or more of the first sampled values only if the first sampled values are within the non-reporting range, and
wherein the weighted moving average includes an exponential weighted moving average d n (w) for an n'th sample of the first sampled values, and wherein the exponential weighted moving average d n (w) is determined from an exponential weighted moving average d n−1 (w) for an (n-1)'th sample and a weight w, where d n (w)=d n w+ d n−1 (1−w) .
21. The method of claim 20 , further comprising discontinuing the act of obtaining second sampled values of the second system metric if the absolute value |cc| is greater than the threshold.
22. The method of claim 20 , further comprising:
reporting one or more of the second sampled values to the system performance monitor if the absolute value |cc| is less than the threshold; and
continuing the act of obtaining second sampled values of the second system metric with at least one agent.
23. The method of claim 20 , wherein the absolute value |cc| is greater than the threshold the method further comprises estimating a value for the second system metric at the system performance monitor by using the correlation coefficient and a plurality of the second sampled values.
24. The method of claim 20 , wherein if the absolute value |cc| is greater than the threshold, the method further comprises assuming at the system performance monitor that values for the second system metric are the same as the first sampled values reported.
25. The method of claim 20 , wherein if the absolute value |cc| is greater than the threshold, the method further comprises estimating a value for the second system metric at the system performance monitor by using the first sampled values reported.
26. A program storage device, readable by a programmable control device, comprising instructions stored on the program storage device for causing the programmable control device to perform a method according to any one of claims 20 through 25 .