Methods and systems that automatically predict distributed-computer-system performance degradation using automatically trained machine-learning components

1. An improved machine-learning-based metric-data processing system that processes metric-data generated within a distributed computer system, the improved machine-learning-based metric-data processing system comprising:

one or more processors;

one or more memories;

one or more data-storage devices; and

computer instructions stored in one or more of the one or more memories that, when executed by one or more of the one or more processors, control the improved machine-learning-based metric-data processing system to

receive multiple metric-data streams from one or more metric-data collection-and-aggregation components of the distributed computer system,

output an n-dimensional metric-value-vector stream to one or more compression and/or storage components;

automatically generate machine-learning-based analysis tools using training data automatically generated from the n-dimensional metric-value-vector stream;

use the automatically generate machine-learning-based analysis tools to monitor the input metric data to detect incipient operational anomalies and problems within the distributed computer system;

report the detected anomalies and problems to management components for amelioration; and

forwarding, by the improved machine-learning-based metric-data processing system, one or more of the automatically generated machine-learning-based analysis tools to management components within the distributed computer system for use in metric-data analysis.

2. The improved machine-learning-based metric-data processing system of claim 1

wherein the multiple metric-data streams each comprises a sequence of encoded metric-data data points, each metric-data data point representable as a timestamp/value pair; and

wherein the value of a timestamp/value pair is one of a scalar value and a vector value.

3. The improved machine-learning-based metric-data processing system of claim 1 wherein the output of the n-dimensional metric-value-vector stream to one or more compression and/or storage components occurs currently and asynchronously with respect to generation of machine-learning-based analysis tools, monitoring of the input metric data, and reporting of the detected anomalies and problems to management components for amelioration.

4. The improved machine-learning-based metric-data processing system of claim 1 wherein automatic generation of training data from the n-dimensional metric-value-vector stream comprises generating a set of logically tabular datasets by:

for each of a set of KPIs,

selecting a subsequence of n-dimensional metric-value vectors from the n-dimensional metric-value-vector stream;

transforming the selected subsequence of n-dimensional metric-value vectors into a logically tabular dataset having metric columns corresponding to elements of the n-dimensional metric-value vectors that each contain a sequence of metric values contained in a particular element of the n-dimensional metric-value vectors;

adding a KPI column to the logically tabular dataset;

pruning the logically tabular dataset so that KPI values in the KPI column are uniformly distributed; and

pruning the metric columns to generate a concise logically tabular dataset.

5. The improved machine-learning-based metric-data processing system of claim 4 wherein adding a KPI column to the logically tabular dataset further comprises:

selecting one or more metric columns of the logically tabular dataset;

from each element in the one or more selected metric columns, generating a numerical KPI value;

transforming the numerical KPI values into categorical KPI values;

adding the KPI column containing the categorical KPI values to the logically tabular dataset; and

removing the selected metric columns from consideration for selection for inclusion in the concise logically tabular dataset.

6. The improved machine-learning-based metric-data processing system of claim 4 wherein pruning the logically tabular dataset so that KPI values in the KPI column are uniformly distributed further comprises:

determining a probability distribution for the categorical KPI values; and

iteratively

considering each row of the logically tabular dataset, and

randomly selecting the considered row for removal from the logically tabular data set with a probability inversely proportional to the probability associated with the categorical KPI values in the determined probability distribution.

7. The improved machine-learning-based metric-data processing system of claim 4 wherein pruning the metric columns to generate a concise logically tabular dataset further comprises:

computing pairwise symmetrical uncertainty values for the metric columns with respect to the KPI column;

selecting metric columns with computed symmetrical uncertainty values less than a threshold value;

computing pairwise symmetrical uncertainty values for selected metric columns with respect to other selected metric columns; and

removing metric columns that are more strongly correlated with other metric columns than with the KPI column.

8. The improved machine-learning-based metric-data processing system of claim 1 wherein automatically generating machine-learning-based analysis tools using automatically generated training data further comprises one of:

generating a set of logical rules from the training data using a rule-induction method; and

generating one or more decision trees from the training data using a decision-tree construction method.

9. The improved machine-learning-based metric-data processing system of claim 1 wherein using the automatically generated machine-learning-based analysis tools to monitor the input metric data to detect incipient operational anomalies and problems within the distributed computer system further comprises one or more of:

applying each automatically generated machine-learning-based analysis tool to a subsequence of n-dimensional metric-value vectors selected from the n-dimensional metric-value-vector stream to generate a categorical KPI value; and

when the generated categorical KPI value is indicative of an operational anomaly or problem, storing an indication of the tool along with the generated categorical KPI value for subsequent reporting.

10. A method, incorporated in an improved machine-learning-based metric-data processing system having one or more processors, one or more memories, one or more data-storage devices, that: automatically generates training data from an n-dimensional metric-value-vector stream; and automatically generates machine-learning-based analysis tools using the automatically generated training data; and using the automatically generated machine-learning-based analysis tools to monitor metric data to detect incipient operational anomalies and problems within the distributed computer system; applying each automatically generated machine-learning-based analysis tool to a subsequence of n-dimensional metric-value vectors selected from the n-dimensional metric-value-vector stream to generate a categorical KPI value; and when the generated categorical KPI value is indicative of an operational anomaly or problem, storing an indication of the tool along with the generated categorical KPI value for subsequent reporting the detected anomalies and problems to management components for amelioration.

11. The method of claim 10 wherein automatically generating training data from the n-dimensional metric-value-vector stream further comprises generating a set of logically tabular datasets by:

for each of a set of KPIs,

selecting a subsequence of n-dimensional metric-value vectors from the n-dimensional metric-value-vector stream;

transforming the selected subsequence of n-dimensional metric-value vectors into a logically tabular dataset having metric columns corresponding to elements of the n-dimensional metric-value vectors that each contain a sequence of metric values contained in a particular element of the n-dimensional metric-value vectors;

adding a KPI column to the logically tabular dataset;

pruning the logically tabular dataset so that KPI values in the KPI column are uniformly distributed; and

pruning the metric columns to generate a concise logically tabular dataset.

12. The method of claim 11 wherein adding a KPI column to the logically tabular dataset further comprises:

selecting one or more metric columns of the logically tabular dataset;

from each element in the one or more selected metric columns, generating a numerical KPI value;

transforming the numerical KPI values into categorical KPI values;

adding the KPI column containing the categorical KPI values to the logically tabular dataset; and

removing the selected metric columns from consideration for selection for inclusion in the concise logically tabular dataset.

13. The method of claim 11 wherein pruning the logically tabular dataset so that KPI values in the KPI column are uniformly distributed further comprises:

determining a probability distribution for the categorical KPI values; and

iteratively

considering each row of the logically tabular dataset, and

randomly selecting the considered row for removal from the logically tabular data set with a probability inversely proportional to the probability associated with the categorical KPI values in the determined probability distribution.

14. The method of claim 11 wherein pruning the metric columns to generate a concise logically tabular dataset further comprises:

computing pairwise symmetrical uncertainty values for the metric columns with respect to the KPI column;

selecting metric columns with computed symmetrical uncertainty values less than a threshold value;

computing pairwise symmetrical uncertainty values for selected metric columns with respect to other selected metric columns; and

removing metric columns that are more strongly correlated with other metric columns than with the KPI column.

15. The method of claim 10 wherein automatically generating machine-learning-based analysis tools using automatically generated training data further comprises one of:

generating a set of logical rules from the training data using a rule-induction method; and

generating one or more decision trees from the training data using a decision-tree construction method.