Method for summarized viewing of large numbers of performance metrics while retaining cognizance of potentially significant deviations
A method is disclosed for determining with computing apparatus an adequate number of clusters for summarizing result data that includes a large number of observation data points. The summary data includes a small number of samples of data from each cluster with the number of clusters being large enough to provide a good summary of all the result data without being so large as to make it difficult for one skilled in the art to examine visually all of the summary data generated by the computing apparatus.
1. A computer-implemented, comprising:
A) determining, by a computing apparatus, an adequate number of clusters for summarizing result data which includes a large number of observation data points each having at least one characterizing value;
B) utilizing, by the computer apparatus, a clustering algorithm to examine a significant plurality of the large number of observation data points, the clustering algorithm allowing for one or more input information parameters including at least a designation of a specified number of clusters that provide to the clustering algorithm the number of clusters into which the results data are to be divided;
C) storing the observation data points with characterizing values in a memory storage device of the computing apparatus;
D) setting the specified number of clusters for the clustering algorithm to be performed by the computing apparatus to a value designating a starting number clusters;
E) clustering the significant plurality of the large number of observation data points into the specified number of clusters utilizing the clustering algorithm performed by the computing apparatus to obtain a specific clustering of the data points for that specified number of clusters such that each of the plurality of the large number of observation data points is a member of one cluster;
F) computing by the computing apparatus, a quality indicator for the specific clustering of the data points from step E), the quality indicator being based on a maximum deviation of a characterizing value of any member of the cluster when compared to a typical value corresponding to a centroid characterizing value of at least a majority of other members of the cluster;
G) storing the quality indicator value and the specified number of clusters for the specific clustering of the data points related to the that specified number of clusters into the memory storage device; and
H) increasing the specified number of clusters and then repeating steps E) through G) two or more times until a preset limit on the specified number of clusters is detected by the computing apparatus or until the computer apparatus makes a selection of one of the already specified number of clusters as the adequate number of clusters based on a pattern of at least two of the already computed quality indicators.
2. The method of claim 1 in which the starting number of clusters is determined the computer apparatus performing a selection of one or more of the following: a) setting the starting number of clusters to a predetermined starting number of clusters, b) setting the starting number of clusters to a value obtained from an input value entered by a user, c) setting the starting number of clusters to value of one, or d) setting the starting number of clusters to a value calculated based on a count of the large number of observation data points.
3. The method of claim 1 in which the starting number of clusters is one or two.
4. The method of claim 1 in which the starting number of clusters is specified by a user of the method as one of the input information parameters.
5. The method of claim 1 in which a minimum number of clusters is specified by a user of the method as one of the input information parameters.
6. The method of claim 1 in which the increasing of the number of clusters is an increasing of the specified number of clusters by one.
7. The method of claim 1 in which the computing apparatus performing the repeating of the steps E) through G) and the increasing of the number of clusters is limited so as to stop upon detecting having reached a predetermined limit on the specified number of clusters.
8. The method of claim 1 in which the step of increasing of the number of clusters is limited so as to stop upon having reached a limit which is based on a count of the large number of observation data points performed by the computing apparatus.
9. The method of claim 1 in which the clustering of step C) performed by the computing apparatus utilizes an algorithm based upon K-Means, K-Means++, or scalable K-Means algorithms.