Methods to associate workloads to optimal system settings based upon statistical models
A system, method, and computer-readable medium are disclosed for optimizing performance of an information handling system comprising: identifying a statistical model for use when optimizing performance of the information handling system; sampling the performance of the information handling system, the sampling being performed iteratively; and, adjusting the performance of the information handling system by applying optimized system configurations to the information handling system, the optimized parameters being based upon the statistical model.
1. A computer-implementable method for optimizing performance of an information handling system comprising:
executing experiments on selected system configurations and system configuration settings on a test system, the experiments comprising applying a set of system configuration settings to the test system and testing performance of the test system when executing known workloads with the set of system configuration settings, the testing performance comprising measuring and characterizing a workload using instrumentation data on how the workload uses a plurality of sub-systems of the test system;
developing a statistical model for use when optimizing the performance of the information handling system, the statistical model being developed based upon the measuring and characterizing the workload, the statistical model being developed to classify workloads, each workload comprising a combination of a single application or multiple applications executed on an information handling system, the characterizing the workload comprising aggregating workload types into clusters of workload types, each of the clusters of workload types representing a number of distinct system configuration settings;
sampling the performance of the information handling system, the sampling being performed iteratively;
adjusting the performance of the information handling system by applying optimized system configurations to the information handling system, the optimized system configurations comprising optimized parameters, the optimized parameters being based upon the statistical model;
characterizing workloads in operation at runtime to analyze the performance of the information handling system;
identifying parameters to be changed to obtain optimal performance based upon the characterizing, optimal performance being an increase in performance of the information handling system when compared with performance of the information handling system without the parameters being changed; and wherein
the aggregation of workload types develops an association of cluster centroids and optimal settings, optimal settings comprising settings which increase performance of the information handling system when compared with performance of the information handling system without application of the optimal settings;
the aggregation of workload types is performed via an aggregation operation, the aggregation operation using non-hierarchical clustering to partition a dataset into clusters, the non-hierarchical clustering comprising K means clustering; and
the K means clustering partitions the dataset into K clusters where K is predefined based on a number of distinct optimal settings, the distinct optimal settings comprising settings which increase performance of the information handling system when compared with performance of the information handling system without application of the distinct optimal settings, the distinct optimal settings comprising at least one of a hyper-threading setting, a Vsync setting and a power saving state setting.
2. The method of claim 1 , wherein:
the adjusting is performed dynamically for varying workloads, the varying workloads comprising random stochastic variation in workload, abrupt user or operating system level discontinuities.
3. The method of claim 2 , wherein:
the workloads being classified comprise known workloads and unknown workloads.
4. The method of claim 1 , wherein:
the statistical model comprises a supervised learning operation, the supervised learning operation statistically mapping system parameters to optimal parameter settings, optimal parameter settings comprising parameter settings which increase performance of the information handling system when compared with performance of the information handling system without application of the optimal parameter settings.
5. The method of claim 1 , further comprising:
reviewing the optimized system configuration to determine whether the optimized parameters have been determined with a sufficient confidence level, the sufficient confidence level comprising a confidence level of greater than 95%.
6. A system comprising:
a processor;
a data bus coupled to the processor; and
a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for:
executing experiments on selected system configurations and system configuration settings on a test system, the experiments comprising applying a set of system configuration settings to the test system and testing performance of the test system when executing known workloads with the set of system configuration settings, the testing performance comprising measuring and characterizing a workload using instrumentation data on how the workload uses a plurality of sub-systems of the test system, the characterizing the workload comprising aggregating workload types into clusters of workload types, each of the clusters of workload types representing a number of distinct optimal system configuration settings;
developing a statistical model for use when optimizing the performance of the information handling system, the statistical model being developed based upon the measuring and characterizing the workload, the statistical model being developed to classify workloads, each workload comprising a combination of a single application or multiple applications executed on an information handling system;
sampling the performance of the information handling system, the sampling being performed iteratively;
adjusting the performance of the information handling system by applying optimized system configurations to the information handling system, the optimized system configurations comprising optimized parameters, the optimized parameters being based upon the statistical model;
characterizing workloads in operation at runtime to analyze the performance of the information handling system;
identifying parameters to be changed to obtain optimal performance based upon the characterizing, optimal performance being an increase in performance of the information handling system when compared with performance of the information handling system without the parameters being changed; and wherein
the aggregation of workload types develops an association of cluster centroids and optimal settings, optimal settings comprising settings which increase performance of the information handling system when compared with performance of the information handling system without application of the optimal settings;
the aggregation of workload types is performed via an aggregation operation, the aggregation operation using non-hierarchical clustering to partition a dataset into clusters, the non-hierarchical clustering comprising K means clustering; and
the K means clustering partitions the dataset into K clusters where K is predefined based on a number of distinct optimal settings, the distinct optimal settings comprising settings which increase performance of the information handling system when compared with performance of the information handling system without application of the distinct optimal settings, the distinct optimal settings comprising at least one of a hyper-threading setting, a Vsync setting and a power saving state setting.
7. The system of claim 6 , wherein:
the adjusting is performed dynamically for varying workloads, the varying workloads comprising random stochastic variation in workload, abrupt user or operating system level discontinuities.
8. The system of claim 6 , wherein:
the workloads being classified comprise known workloads and unknown workloads.
9. The system of claim 6 , wherein:
the statistical model comprises a supervised learning operation, the supervised learning operation statistically mapping system parameters to optimal parameter settings.
10. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
executing experiments on selected system configurations and system configuration settings on a test system, the experiments comprising applying a set of system configuration settings to the test system and testing performance of the test system when executing known workloads with the set of system configuration settings, the testing performance comprising measuring and characterizing a workload using instrumentation data on how the workload uses a plurality of sub-systems of the test system;
developing a statistical model for use when optimizing the performance of the information handling system, the statistical model being developed based upon the measuring and characterizing the workload, the statistical model being developed to classify workloads, each workload comprising a combination of a single application or multiple applications executed on an information handling system, the characterizing the workload comprising aggregating workload types into clusters of workload types, each of the clusters of workload types representing a number of distinct system configuration settings;
sampling the performance of the information handling system, the sampling being performed iteratively;
adjusting the performance of the information handling system by applying optimized system configurations to the information handling system, the optimized system configurations comprising optimized parameters, the optimized parameters being based upon the statistical model;
characterizing workloads in operation at runtime to analyze the performance of the information handling system;
identifying parameters to be changed to obtain optimal performance based upon the characterizing, optimal performance being an increase in performance of the information handling system when compared with performance of the information handling system without the parameters being changed; and wherein
the aggregation of workload types develops an association of cluster centroids and optimal settings, optimal settings comprising settings which increase performance of the information handling system when compared with performance of the information handling system without application of the optimal settings;
the aggregation of workload types is performed via an aggregation operation, the aggregation operation using non-hierarchical clustering to partition a dataset into clusters, the non-hierarchical clustering comprising K means clustering; and
the K means clustering partitions the dataset into K clusters where K is predefined based on a number of distinct optimal settings, the distinct optimal settings comprising settings which increase performance of the information handling system when compared with performance of the information handling system without application of the distinct optimal settings, the distinct optimal settings comprising at least one of a hyper-threading setting, a Vsync setting and a power saving state setting.
11. The non-transitory, computer-readable storage medium of claim 10 , wherein:
the adjusting is performed dynamically for varying workloads, the varying workloads comprising random stochastic variation in workload, abrupt user or operating system level discontinuities.
12. The non-transitory, computer-readable storage medium of claim 10 , wherein:
the workloads being classified comprise known workloads and unknown workloads.
13. The non-transitory, computer-readable storage medium of claim 10 , wherein:
the statistical model comprises a supervised learning operation, the supervised learning operation statistically mapping system parameters to optimal parameter settings.