Finding outliers in similar time series samples
A method, computer program product, and computer system for finding outliers in multi-dimensional time series samples. Each time series sample is divided into at least 2 sub samples having equal time duration. At least one prediction model is pre-trained using the sub samples and a prediction result for each sub sample for each prediction model is obtained by executing the pre-trained prediction models with the time series samples as input. A Shapely value corresponding to each prediction result is sub samples for each prediction model to generate multiple clusters of Shapely values for each prediction model. Highest ranking Shapely value outliers are determined from analysis of the multiple clusters. Highest ranking outlier sub samples corresponding to the highest ranking Shapely value outliers are identified.
1 . A method for finding outliers in multi-dimensional time series samples, the method comprising:
dividing, by one or more processors of a computer system, each time series sample of N time series samples into at least 2 sub samples having equal time duration, wherein N is at least 2, wherein each time series sample is a M-dimensional time series sample having M dimensions, wherein M is at least 2, wherein each dimension of the M dimensions is a different attribute expressed as a function of time by each time series sample, wherein each sub sample is an M-dimensional sub sample having the M dimensions;
pre-training, by the one or more processors, each prediction model of L prediction models using the M-dimensional sub samples and obtaining, by the one or more processors from executing the L pre-trained prediction models with the N time series samples as input, an M-dimensional prediction result for each M-dimensional sub sample for each prediction model, wherein L is at least 1;
computing, by the one or more processors, a M-dimensional Shapely value corresponding to each M-dimensional prediction result for each M-dimensional sub sample for each prediction model, wherein each dimension of the M-dimensional Shapely value corresponds to a respective dimension of each M-dimensional time series sample;
clustering, by the one or more processors, the M-dimensional Shapely values for the M-dimensional sub samples for each prediction model to generate multiple clusters of M-dimensional Shapely values for each prediction model;
determining, by the one or more processors from analysis of the multiple clusters of the L prediction models, H highest ranking M-dimensional Shapely value outliers, wherein each of the H highest ranking M-dimensional Shapely value outliers is specific to one dimension of the M dimensions, wherein His at least 1;
identifying, by the one or more processors, H outlier sub samples corresponding to the H highest ranking M-dimensional Shapely value outliers, wherein each of the H outlier sub samples is specific to the one dimension; and
aligning, by the one or more processors and based on performing dynamic time warping (DTW), the N time series samples at each of the H outlier sub samples to obtain N aligned sub samples for each of the H outlier sub samples.
2 . The method of claim 1 , the method further comprising:
dividing, by the one or more processors, each aligned sub sample of the N aligned sub samples into multiple time windows, wherein a time duration of each time window is a monotonically decreasing function of a volatility of attributes in each time window;
clustering, by the one or more processors, the attributes in each time window to generate a plurality of clusters for the attributes in each time window; and
determining, by the one or more processors from analysis of the plurality of clusters for the attributes in each time window, h highest ranking outlier time windows, wherein each of the h highest ranking outlier time windows is specific to a particular dimension of the M dimensions, sand wherein h is at least 1.
3 . The method of claim 2 ,
wherein the volatility is a standard deviation of attribute values in each time window, and wherein the time duration of each time window varies inversely with the standard deviation.
4 . The method of claim 2 ,
wherein the volatility is a variance of attribute values in each time window, and wherein the time duration of each time window varies inversely with the variance.
5 . The method of claim 2 ,
wherein the clustering the attributes in each time window comprises clustering the attributes in each time window using k clustering algorithms to generate respective k groups of clusters in each time window representing the plurality of clusters, wherein k is at least 2; and
wherein the determining the h highest ranking outlier time windows comprises voting, by the k clustering algorithms, a ranking of outlier attribute values in the time windows to determine the h highest ranking outlier time windows.
6 . The method of claim 1 ,
wherein the clustering the M-dimensional Shapely values comprises clustering the M-dimensional Shapely values using K clustering algorithms to generate respective K groups of clusters, wherein K is at least 2; and
wherein the determining the H highest ranking M-dimensional Shapely value outliers comprises voting, by the K clustering algorithms, a ranking of M-dimensional Shapely value outliers to determine the H highest ranking M-dimensional Shapely value outliers.
7 . The method of claim 1 ,
wherein L is at least 2.
8 . The method of claim 1 ,
wherein performing the DTW is associated with aligning wave shapes of the N time series samples.
9 . A computer program product, comprising one or more computer readable hardware storage devices having computer readable program code stored therein, the program code containing instructions executable by one or more processors of a computer system to implement a method for finding outliers in multi-dimensional time series samples, the method comprising:
dividing, by the one or more processors, each time series sample of N time series samples into at least 2 sub samples having equal time duration, wherein N is at least 2, wherein each time series sample is a M-dimensional time series sample having M dimensions, wherein M is at least 2, wherein each dimension of the M dimensions is a different attribute expressed as a function of time by each time series sample, wherein each sub sample is an M-dimensional sub sample having the M dimensions;
pre-training, by the one or more processors, each prediction model of L prediction models using the M-dimensional sub samples and obtaining, by the one or more processors from executing the L pre-trained prediction models with the N time series samples as input, an M-dimensional prediction result for each M-dimensional sub sample for each prediction model, wherein L is at least 1;
computing, by the one or more processors, a M-dimensional Shapely value corresponding to each M-dimensional prediction result for each M-dimensional sub sample for each prediction model, wherein each dimension of the M-dimensional Shapely value corresponds to a respective dimension of each M-dimensional time series sample;
clustering, by the one or more processors, the M-dimensional Shapely values for the M-dimensional sub samples for each prediction model to generate multiple clusters of M-dimensional Shapely values for each prediction model;
determining, by the one or more processors from analysis of the multiple clusters of the L prediction models, H highest ranking M-dimensional Shapely value outliers, wherein each of the H highest ranking M-dimensional Shapely value outliers is specific to one dimension of the M dimensions, wherein His at least 1;
identifying, by the one or more processors, H outlier sub samples corresponding to the H highest ranking M-dimensional Shapely value outliers, wherein each of the H outlier sub samples is specific to the one dimension; and
aligning by the one or more processors and based on performing dynamic time warping (DTW), the N time series samples at each of the H outlier sub samples to obtain N aligned sub samples for each of the H outlier sub samples.
10 . The computer program product of claim 9 , the method further comprising:
dividing, by the one or more processors, each aligned sub sample of the N aligned sub samples into multiple time windows, wherein a time duration of each time window is a monotonically decreasing function of a volatility of attributes in each time window;
clustering, by the one or more processors, the attributes in each time window to generate a plurality of clusters for the attributes in each time window; and
determining, by the one or more processors from analysis of the plurality of clusters for the attributes in each time window, h highest ranking outlier time windows, wherein each of the h highest ranking outlier time windows is specific to a particular dimension of the M dimensions, sand wherein h is at least 1.
11 . The computer program product of claim 10 ,
wherein the volatility is a standard deviation of attribute values in each time window, and wherein the time duration of each time window varies inversely with the standard deviation.
12 . The computer program product of claim 10 ,
wherein the volatility is a variance of attribute values in each time window, and wherein the time duration of each time window varies inversely with the variance.
13 . The computer program product of claim 10 ,
wherein the clustering the attributes in each time window comprises clustering the attributes in each time window using k clustering algorithms to generate respective k groups of clusters in each time window representing the plurality of clusters, wherein k is at least 2; and
wherein the determining the h highest ranking outlier time windows comprises voting, by the k clustering algorithms, a ranking of outlier attribute values in the time windows to determine the h highest ranking outlier time windows.
14 . The computer program product of claim 9 ,
wherein the clustering the M-dimensional Shapely values comprises clustering the M-dimensional Shapely values using K clustering algorithms to generate respective K groups of clusters, wherein K is at least 2; and
wherein the determining the H highest ranking M-dimensional Shapely value outliers comprises voting, by the K clustering algorithms, a ranking of M-dimensional Shapely value outliers to determine the H highest ranking M-dimensional Shapely value outliers.
15 . A computer system, comprising one or more processors, one or more memories, and one or more computer readable hardware storage devices, the one or more hardware storage devices containing program code executable by the one or more processors via the one or more memories to implement a method for finding outliers in multi-dimensional time series samples, the method comprising:
dividing, by the one or more processors, each time series sample of N time series samples into at least 2 sub samples having equal time duration, wherein N is at least 2, wherein each time series sample is a M-dimensional time series sample having M dimensions, wherein M is at least 2, wherein each dimension of the M dimensions is a different attribute expressed as a function of time by each time series sample, wherein each sub sample is an M-dimensional sub sample having the M dimensions;
pre-training, by the one or more processors, each prediction model of L prediction models using the M-dimensional sub samples and obtaining, by the one or more processors from executing the L pre-trained prediction models with the N time series samples as input, an M-dimensional prediction result for each M-dimensional sub sample for each prediction model, wherein L is at least 1;
computing, by the one or more processors, a M-dimensional Shapely value corresponding to each M-dimensional prediction result for each M-dimensional sub sample for each prediction model, wherein each dimension of the M-dimensional Shapely value corresponds to a respective dimension of each M-dimensional time series sample;
clustering, by the one or more processors, the M-dimensional Shapely values for the M-dimensional sub samples for each prediction model to generate multiple clusters of M-dimensional Shapely values for each prediction model;
determining, by the one or more processors from analysis of the multiple clusters of the L prediction models, H highest ranking M-dimensional Shapely value outliers, wherein each of the H highest ranking M-dimensional Shapely value outliers is specific to one dimension of the M dimensions, wherein His at least 1;
identifying, by the one or more processors, H outlier sub samples corresponding to the H highest ranking M-dimensional Shapely value outliers, wherein each of the H outlier sub samples is specific to the one dimension; and
aligning by the one or more processors and based on performing dynamic time warping (DTW), the N time series samples at each of the H outlier sub samples to obtain N aligned sub samples for each of the H outlier sub samples.
16 . The computer system of claim 15 , the method further comprising:
dividing, by the one or more processors, each aligned sub sample of the N aligned sub samples into multiple time windows, wherein a time duration of each time window is a monotonically decreasing function of a volatility of attributes in each time window;
clustering, by the one or more processors, the attributes in each time window to generate a plurality of clusters for the attributes in each time window; and
determining, by the one or more processors from analysis of the plurality of clusters for the attributes, h highest ranking outlier time windows, wherein each of the h highest ranking outlier time windows is specific to a particular dimension of the M dimensions, sand wherein h is at least 1.
17 . The computer system of claim 16 ,
wherein the volatility is a standard deviation of attribute values in each time window, and wherein the time duration of each time window varies inversely with the standard deviation.
18 . The computer system of claim 16 ,
wherein the volatility is a variance of attribute values in each time window, and wherein the time duration of each time window varies inversely with the variance.
19 . The computer system of claim 16 ,
wherein the clustering the attributes in each time window comprises clustering the attributes in each time window using k clustering algorithms to generate respective k groups of clusters in each time window representing the plurality of clusters, wherein k is at least 2; and
wherein the determining the h highest ranking outlier time windows comprises voting, by the k clustering algorithms, a ranking of outlier attribute values in the time windows to determine the h highest ranking outlier time windows.
20 . The computer system of claim 15 ,
wherein the clustering the M-dimensional Shapely values comprises clustering the M-dimensional Shapely values using K clustering algorithms to generate respective K groups of clusters, wherein K is at least 2; and
wherein the determining the H highest ranking M-dimensional Shapely value outliers comprises voting, by the K clustering algorithms, a ranking of M-dimensional Shapely value outliers to determine the H highest ranking M-dimensional Shapely value outliers.