Resource and metric ranking by differential analysis
The technology disclosed relates to differential analysis of sets of time series pairs. In particular, it relates to building estimators of magnitude of difference between two time series. After the basic estimators are built, they are combined into ensemble estimators using linear or nonlinear prediction models to improve their accuracy. In one application, the ensemble is used for estimating the magnitudes of difference over sets of metric pairs observed from distributed applications and systems running over a computer network. The metric pairs are then ranked in decreasing order of magnitude of difference to guide an operator in prioritizing his root cause analysis of faults, thereby reducing the time-to-resolution of problems.
1. A method of improving performance of a network under monitoring, the method including:
accumulating, for a resource running on the network, data for a performance metric being applied to monitor the network, the data including a reference time series and a query time series, wherein the reference time series represents the performance metric during a historic time period and the query time series represents the performance metric during a current time period;
transforming the reference time series and the query time series by:
segmenting each time series into a sub-sequence of overlapping or non-overlapping windows with time stamped boundaries;
using a sub-sequence comparison function to compare adjacent window pairs and producing, for each window pair, a statistic or a significance value as a result of the comparison; and
aligning the adjacent window pairs determined for the reference time series with the adjacent window pairs determined for the query time series by attaching, to respective statistics or significance values, a time stamp of one of the time stamped boundaries;
comparing the transformed reference and query time series using cross-correlation to determine a magnitude of difference between the reference and query time series; and
based on the magnitude of difference determined, determining whether the resource is undergoing anomalous performance and whenever a resource is identified as undergoing anomalous performance, raising a signal that identifies anomalous performance for further action.
2. The method of claim 1 , wherein the reference and query time series are reference and query time periods of a single time series.
3. The method of claim 2 , wherein the single time series is a performance metric collected from a resource.
4. The method of claim 1 , further including comparing the transformed reference and query time series by recording one minus a normalized cross-correlation and estimating the magnitude of difference between the reference and query time series.
5. The method of claim 1 , wherein the sub-sequence comparison function is Kolmogorov-Smirnov (KS) test.
6. The method of claim 1 , further including aligning the reference window pairs with the query window pairs by attaching, to the respective statistics or significance values, a time stamp of a right time stamped boundary of a right window or a time stamp of a left time stamped boundary of a left window or a time stamp of a left time stamped boundary of a right window or a time stamp of a right time stamped boundary of a left window, where time increases from left to right.
7. The method of claim 1 , further including comparing the transformed reference and query time series by determining a dynamic-time-warping (DTW) distance and estimating the magnitude of difference between the reference and query time series.
8. The method of claim 1 , further including applying the cross-correlation after smoothing the transformed reference and query time series by convolving with a square or a Gaussian kernel.
9. The method of claim 1 , wherein raising a signal includes raising an exception interrupting a normal flow of processing at a monitoring terminal.
10. The method of claim 1 , wherein further action includes rerouting traffic, obtaining further inspection; automatically searching for and applying a service update to the resource or automatically restarting the resource.
11. A method of combining basic estimates of magnitudes of difference between two time series of metrics into an improved estimate, the method including:
sampling a training sample of resources and, for each sampled resource, sampling a set of time series along reference and query time periods;
receiving ideal magnitudes of difference estimates as target scores;
fitting a regression model to the target scores using the basic estimates of the magnitudes of difference; and
storing output of the regression model as the improved estimate; and
wherein additional regression variables are generated using square and cross-product functions on the basic estimates of the magnitudes of difference by:
estimating a power spectrum density (PSD) of each of the time series;
estimating a total power in each time series;
dividing a frequency axis in to three regions as low, medium, and high frequencies;
estimating ratio of power in each of the three regions to the total power of the time series; and
generating at least six regression variables for each of the time series as low, medium, and high frequency power fraction of the reference and query periods.
12. The method of claim 11 , further including summarizing the magnitudes of difference of a resource by averaging over the magnitudes of difference.
13. The method of claim 12 , further including summarizing by determining an order statistic such as a median, maximum, or minimum.
14. The method of claim 11 , wherein the training sample includes 10 resources and 12 metrics per resource.
15. The method of claim 11 further including receiving an ideal ordering of the metrics for each resource separately used as target orderings for fitting an ordinal regression model using the additional regression variables.
16. The method of claim 11 , wherein the regression model is a linear model.
17. The method of claim 11 , wherein the regression model is a nonlinear model, including a decision tree or a gradient boosted regression model.
18. A system including one or more processors coupled to memory, the memory loaded with computer instructions to improve performance of a monitored network by determining a magnitude of difference between a reference time series and a query time series, the instructions, when executed on the processors, implement actions comprising:
accumulating, for a resource running on the network, data for a performance metric being applied to monitor the network, the data including a reference time series and a query time series, wherein the reference time series represents the performance metric during a historic time period and the query time series represents the performance metric during a current time period;
transforming the reference time series and the query time series by:
segmenting each time series into a sub-sequence of overlapping or non-overlapping windows with time stamped boundaries;
using a sub-sequence comparison function to compare adjacent window pairs and producing, for each window pair, a statistic or a significance value as a result of the comparison; and
aligning the adjacent window pairs determined for the reference time series with the adjacent window pairs determined for the query time series by attaching, to respective statistics or significance values, a time stamp of one of the time stamped boundaries;
comparing the transformed reference and query time series using cross-correlation to determine a magnitude of difference between the reference and query time series; and
based on the magnitude of difference determined, determining whether the resource is undergoing anomalous performance and whenever a resource is identified as undergoing anomalous performance, raising a signal that identifies anomalous performance for further action.
19. A non-transitory computer readable storage medium impressed with computer program instructions to improve performance of a monitored network by determining a magnitude of difference between a reference time series and a query time series, the instructions, when executed on a processor, implement a method comprising:
accumulating, for a resource running on the network, data for a performance metric being applied to monitor the network, the data including a reference time series and a query time series, wherein the reference time series represents the performance metric during a historic time period and the query time series represents the performance metric during a current time period;
transforming the reference time series and the query time series by:
segmenting each time series into a sub-sequence of overlapping or non-overlapping windows with time stamped boundaries;
using a sub-sequence comparison function to compare adjacent window pairs and producing, for each window pair, a statistic or a significance value as a result of the comparison; and
aligning the adjacent window pairs determined for the reference time series with the adjacent window pairs determined for the query time series by attaching, to respective statistics or significance values, a time stamp of one of the time stamped boundaries;
comparing the transformed reference and query time series using cross-correlation to determine a magnitude of difference between the reference and query time series; and
based on the magnitude of difference determined, determining whether the resource is undergoing anomalous performance and whenever a resource is identified as undergoing anomalous performance, raising a signal that identifies anomalous performance for further action.
20. A system including one or more processors coupled to memory, the memory loaded with computer instructions to combine basic estimates of magnitudes of difference between two time series into an improved estimate, the instructions, when executed on the processors, implement actions of:
sampling a training sample of resources and, for each sampled resource, sampling a set of time series along reference and query time periods;
receiving ideal magnitudes of difference estimates as target scores;
fitting a regression model to the target scores using the basic estimates of the magnitudes of difference; and
storing output of the regression model as the improved estimate; and
wherein additional regression variables are generated using square and cross-product functions on the basic estimates of the magnitudes of difference by:
estimating a power spectrum density (PSD) of each of the time series;
estimating a total power in each time series;
dividing a frequency axis in to three regions as low, medium, and high frequencies;
estimating ratio of power in each of the three regions to the total power of the time series; and
generating at least six regression variables for each of the time series as low, medium, and high frequency power fraction of the reference and query periods.