Statistical detection of site speed performance anomalies
A system for processing data is provided. During operation, the system obtains a current window of one or more intervals of timeseries data collected from a monitored system. Next, the system continuously performs a statistical hypothesis test that compares the one or more intervals of the time-series data with baseline values from historic time-series data associated with the monitored system. When the statistical hypothesis test indicates a deviation of the time-series data from the baseline values, the system outputs an alert of an anomaly represented by the deviation.
1. A method, comprising:
obtaining a current window of one or more intervals of time-series data collected from a monitored system;
repeatedly performing, by a computer system, a statistical hypothesis test that compares the one or more intervals of the time-series data with baseline values from historic time-series data associated with the monitored system; and
when the statistical hypothesis test indicates a deviation of the time-series data from the baseline values:
transforming the baseline values to generate one or more severity levels associated with an anomaly represented by the deviation;
repeating the statistical hypothesis test with the transformed baseline values to identify a severity of the anomaly; and
outputting an alert of the anomaly.
2. The method of claim 1 , further comprising:
generating the baseline values from the historic time-series data based on a seasonality of the time-series data.
3. The method of claim 2 , wherein generating the baseline values from the historic time-series data based on the seasonality of the time-series data comprises:
obtaining one or more previous windows of the historic time-series data from one or more seasonal periods prior to a current seasonal period that contains the current window; and
aggregating the historic time-series data from the one or more previous windows into one or more additional intervals that correspond to the one or more intervals of the time-series data within the current seasonal period and the current window.
4. The method of claim 1 , further comprising:
including the severity of the anomaly in the outputted alert.
5. The method of claim 1 , wherein obtaining the one or more intervals of the time-series data collected during the execution of the monitored system comprises:
aggregating the time-series data within the one or more intervals.
6. The method of claim 5 , wherein the aggregated time-series data comprises at least one of:
a median;
a mean;
a quantile;
a variance; and
a count.
7. The method of claim 1 , wherein the statistical hypothesis test comprises a sign test.
8. The method of claim 1 , wherein the time-series data comprises a page loading time.
9. The method of claim 8 , wherein outputting the alert of the anomaly represented by the deviation comprises at least one of:
transmitting the alert to a page owner of a web page associated with the page loading time; and
transmitting the alert to an infrastructure owner associated with a location of the anomaly.
10. The method of claim 1 , wherein outputting the alert of the anomaly represented by the deviation comprises:
matching one or more attributes of the anomaly to the alert; and
grouping one or more additional anomalies into the alert with the anomaly.
11. An apparatus, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
obtain a current window of one or more intervals of time-series data collected from a monitored system;
perform a statistical hypothesis test that compares the one or more intervals of the time-series data with baseline values from historic time-series data associated with the monitored system; and
when the statistical hypothesis test indicates a deviation of the time-series data from the baseline values:
transform the baseline values to generate one or more severity levels associated with an anomaly represented by the deviation;
repeat the statistical hypothesis test with the transformed baseline values to identify a severity of the anomaly; and
output an alert of the anomaly.
12. The apparatus of claim 11 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
generate the baseline values from the historic time-series data based on a seasonality of the time-series data.
13. The apparatus of claim 12 , wherein generating the baseline values from the historic time-series data based on the seasonality of the time-series data comprises:
obtaining one or more previous windows of the historic time-series data from one or more seasonal periods prior to a current seasonal period that contains the current window; and
aggregating the historic time-series data from the one or more previous windows into one or more additional intervals that correspond to the one or more intervals of the time-series data within the current seasonal period and the current window.
14. The apparatus of claim 11 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
include the severity of the anomaly in the outputted alert.
15. The apparatus of claim 11 , wherein obtaining the one or more intervals of the time-series data collected during the execution of the monitored system comprises:
aggregating the time-series data within the one or more intervals.
16. The apparatus of claim 15 , wherein the aggregated time-series data comprises at least one of:
a median;
a mean;
a quantile;
a variance; and
a count.
17. The apparatus of claim 11 , wherein the statistical hypothesis test comprises a sign test.
18. A system, comprising:
an analysis module comprising a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the system to:
obtain a current window of one or more intervals of time-series data collected from a monitored system; and
perform a statistical hypothesis test that compares the one or more intervals of the time-series data with baseline values from historic time-series data associated with the monitored system; and
a management module comprising a non-transitory computer-readable medium comprising instructions that, when executed by the one or more processors, cause the system to, when the statistical hypothesis test indicates a deviation of the time-series data from the baseline values:
transform the baseline values to generate one or more severity levels associated with an anomaly represented by the deviation;
repeat the statistical hypothesis test with the transformed baseline values to identify a severity of the anomaly; and
output an alert of the anomaly.
19. The system of claim 18 , wherein the non-transitory computer-readable medium of the analysis module further comprises instructions that, when executed by the one or more processors, cause the system to:
generate the baseline values from the historic time-series data based on a seasonality of the time-series data.
20. The system of claim 18 , wherein the non-transitory computer-readable medium of the analysis module further comprises instructions that, when executed by the one or more processors, cause the system to:
include the severity of the anomaly in the outputted alert.