Run-time determination of application performance with low overhead impact on system performance

1. A method, comprising:

performing, by a system configuration tool executing on a computing system, an automated process to determine a run-time performance of an application executing on the computing system;

wherein performing the automated process comprises:

obtaining a time series telemetry data stream for each of a plurality of key performance indicators corresponding to utilization of resources of the computing system by the application during run-time execution of the application on the computing system with the computing system having a first system configuration;

populating model parameters of a trained machine learning performance score model with parameter values that are determined based on the obtained time series telemetry data streams, wherein at least one model parameter comprises a learned weight value, and a learned feature which specifies (i) an associated key performance indicator, and (ii) an associated statistical feature extraction operation to perform on a time series telemetry data stream associated with the given key performance indicator to compute a value for the learned feature, wherein a parameter value of the at least one model parameter is computed by applying the associated statistical feature extraction operation to at least a portion of the obtained time series telemetry data stream associated with the given key performance indicator to determine the value of the learned feature, and applying the learned weight value to the determined value of the learned feature;

determining a run-time performance score of the application using the model parameters of the trained machine learning performance score model populated with the parameter values based on the obtained time series telemetry data streams; and

in response to the determined run-time performance score of the application, automatically determining whether to one of: (i) maintain the first system configuration and (ii) adjust at least one resource configuration setting of the computing system to alter a run-time performance of the application, based on the determined run-time performance score of the application;

wherein the trained machine learning performance score model comprises one or more storage performance models in which model parameters of the one or more storage performance models are trained to determine one or more storage performance metrics.

2. The method of claim 1 , wherein the automated process further comprises persistently storing the determined run-time performance score of the application in a data structure that maps the determined run-time performance score to information regarding the first system configuration of the computing system.

3. The method of claim 1 , wherein the trained machine learning performance score model comprises an application agnostic and system configuration agnostic machine learning performance score model, and wherein the one or more storage performance metrics comprise one or more of: (i) average latency for read operations; (ii) average latency of write operations; (iii) bandwidth for read operations; (iv) bandwidth of write operations; (v) input/output operations per second for write operations; and (vi) input/output operations per second for read operations.

4. The method of claim 1 , wherein applying the associated statistical feature extraction operation to at least a portion of the obtained time series telemetry data stream associated with the given key performance indicator, comprises:

computing summary statistics on telemetry data sample values of the obtained time series telemetry data stream associated with the given key performance indicator;

wherein the summary statistics comprise one or more of: (i) mean; (ii) standard deviation; (iii) minimum; (iv) maximum; (v) 25 th percentile; (vi) 50 th percentile; and (vii) 75 th percentile.

5. The method of claim 1 , wherein determining the run-time performance score of the application using the model parameters of the trained machine learning performance score model populated with the parameter values based on the obtained time series telemetry data streams, comprises:

computing a parameter value for each model parameter of the trained machine learning performance score model; and

determining a sum of the computed parameter values of the model parameters of the trained machine learning performance score model;

wherein the determined sum corresponds to a performance score for a given storage performance metric defined by the trained machine learning performance score model.

6. The method of claim 1 , wherein the time series telemetry data streams for the plurality of key performance indicators are obtained using one or more of performance counters that execute on the computing system, wherein the performance counters comprise one or more of: (i) system performance counters; (ii) physical disk performance counters; (iii) logical disk performance counters; (iv) process performance counters; and (v) cache performance counters.

7. The method of claim 1 , wherein the automated process further comprises:

determining a second run-time performance score of the application using the model parameters of the trained machine learning performance score model populated with values of extracted statistical features obtained during run-time execution of the application on the computing system with the computing system having a second system configuration, which is different from the first system configuration; and

comparing the second run-time performance score with a previously determined run-time performance score of the application executing on the computing system having the first system configuration; and

determining which of the first and second system configurations of the computing system provide a greater run-time performance of the application based on said comparison of the run-time performance scores.

8. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code is executable by one or more processors to perform a method which comprises:

performing, by a system configuration tool executing on a computing system, an automated process to determine a run-time performance of an application executing on the computing system;

wherein performing the automated process comprises:

obtaining a time series telemetry data stream for each of a plurality of key performance indicators corresponding to utilization of resources of the computing system by the application during run-time execution of the application on the computing system with the computing system having a first system configuration;

populating model parameters of a trained machine learning performance score model with parameter values that are determined based on the obtained time series telemetry data streams, wherein at least one model parameter comprises a learned weight value, and a learned feature which specifies (i) an associated key performance indicator, and (ii) an associated statistical feature extraction operation to perform on a time series telemetry data stream associated with the given key performance indicator to compute a value for the learned feature, wherein a parameter value of the at least one model parameter is computed by applying the associated statistical feature extraction operation to at least a portion of the obtained time series telemetry data stream associated with the given key performance indicator to determine the value of the learned feature, and applying the learned weight value to the determined value of the learned feature;

determining a run-time performance score of the application using the model parameters of the trained machine learning performance score model populated with the parameter values based on the obtained time series telemetry data streams; and

in response to the determined run-time performance score of the application, automatically determining whether to one of: (i) maintain the first system configuration and (ii) adjust at least one resource configuration setting of the computing system to alter a run-time performance of the application, based on the determined run-time performance score of the application;

wherein the trained machine learning performance score model comprises one or more storage performance models in which model parameters of the one or more storage performance models are trained to determine one or more storage performance metrics.

9. The article of manufacture of claim 8 , further comprising program code that is executable by the one or more processors to perform a method comprising persistently storing the determined run-time performance score of the application in a data structure that maps the determined run-time performance score to information regarding the first system configuration of the computing system.

10. The article of manufacture of claim 8 , wherein the trained machine learning performance score model comprises an application agnostic and system configuration agnostic machine learning performance score model, and wherein the one or more storage performance metrics comprise one or more of: (i) average latency for read operations; (ii) average latency of write operations; (iii) bandwidth for read operations; (iv) bandwidth of write operations; (v) input/output operations per second for write operations; and (vi) input/output operations per second for read operations.

11. The article of manufacture of claim 8 , wherein the program code for applying the associated statistical feature extraction operation to at least a portion of the obtained time series telemetry data stream associated with the given key performance indicator, comprises program code for:

computing summary statistics on telemetry data sample values of the obtained time series telemetry data stream associated with the given key performance indicator;

wherein the summary statistics comprise one or more of: (i) mean; (ii) standard deviation; (iii) minimum; (iv) maximum; (v) 25 th percentile; (vi) 50 th percentile; and (vii) 75 th percentile.

12. The article of manufacture of claim 8 , wherein the program code for determining the run-time performance score of the application using the model parameters of the trained machine learning performance score model populated with the parameter values based on the obtained time series telemetry data streams, comprises program code for:

computing a parameter value for each model parameter of the trained machine learning performance score model; and

determining a sum of the computed parameter values of the model parameters of the trained machine learning performance score model;

wherein the determined sum corresponds to a performance score for a given storage performance metric defined by the trained machine learning performance score model.

13. The article of manufacture of claim 8 , wherein the time series telemetry data streams for the plurality of key performance indicators are obtained using one or more of performance counters that execute on the computing system, wherein the performance counters comprise one or more of: (i) system performance counters; (ii) physical disk performance counters; (iii) logical disk performance counters; (iv) process performance counters; and (v) cache performance counters.

14. A computing system, comprising:

memory to store software instructions;

one or more processors that execute the software instructions to instantiate a system configuration tool which is configured to perform an automated process to determine a run-time performance of an application executing on the computing system, wherein the automated process comprises:

obtaining a time series telemetry data stream for each of a plurality of key performance indicators corresponding to utilization of resources of the computing system by the application during run-time execution of the application on the computing system with the computing system having a first system configuration;

populating model parameters of a trained machine learning performance score model with parameter values that are determined based on the obtained time series telemetry data streams, wherein at least one model parameter comprises a learned weight value, and a learned feature which specifies (i) an associated key performance indicator, and (ii) an associated statistical feature extraction operation to perform on a time series telemetry data stream associated with the given key performance indicator to compute a value for the learned feature, wherein a parameter value of the at least one model parameter is computed by applying the associated statistical feature extraction operation to at least a portion of the obtained time series telemetry data stream associated with the given key performance indicator to determine the value of the learned feature, and applying the learned weight value to the determined value of the learned feature;

determining a run-time performance score of the application using the model parameters of the trained machine learning performance score model populated with the parameter values based on the obtained time series telemetry data streams; and

in response to the determined run-time performance score of the application, automatically determining whether to one of: (i) maintain the first system configuration and (ii) adjust at least one resource configuration setting of the computing system to alter a run-time performance of the application, based on the determined run-time performance score of the application;

wherein the trained machine learning performance score model comprises one or more storage performance models in which model parameters of the one or more storage performance models are trained to determine one or more storage performance metrics.

15. The computing system of claim 14 , wherein the trained machine learning performance score model comprises an application agnostic and system configuration agnostic machine learning performance score model, and wherein the one or more storage performance metrics comprise one or more of: (i) average latency for read operations; (ii) average latency of write operations; (iii) bandwidth for read operations; (iv) bandwidth of write operations; (v) input/output operations per second for write operations; and (vi) input/output operations per second for read operations.

16. The computing system of claim 14 , wherein applying the associated statistical feature extraction operation to at least a portion of the obtained time series telemetry data stream associated with the given key performance indicator, comprises:

computing summary statistics on telemetry data sample values of the obtained time series telemetry data stream associated with the given key performance indicator;

wherein the summary statistics comprise one or more of: (i) mean; (ii) standard deviation; (iii) minimum; (iv) maximum; (v) 25 th percentile; (vi) 50 th percentile; and (vii) 75 th percentile.

17. The computing system of claim 14 , determining the run-time performance score of the application using the model parameters of the trained machine learning performance score model populated with the parameter values based on the obtained time series telemetry data streams, comprises:

computing a parameter value for each model parameter of the trained machine learning performance score model; and

determining a sum of the computed parameter values of the model parameters of the trained machine learning performance score model;

wherein the determined sum corresponds to a performance score for a given storage performance metric defined by the trained machine learning performance score model.

18. The computing system of claim 14 , wherein the time series telemetry data streams for the plurality of key performance indicators are obtained using one or more of performance counters that execute on the computing system, wherein the performance counters comprise one or more of: (i) system performance counters; (ii) physical disk performance counters; (iii) logical disk performance counters; (iv) process performance counters; and (v) cache performance counters.

19. The computing system of claim 14 , wherein the automated process further comprises:

determining a second run-time performance score of the application using the model parameters of the trained machine learning performance score model populated with values of extracted statistical features obtained during run-time execution of the application on the computing system with the computing system having a second system configuration, which is different from the first system configuration; and

comparing the second run-time performance score with a previously determined run-time performance score of the application executing on the computing system having the first system configuration; and

determining which of the first and second system configurations of the computing system provide a greater run-time performance of the application based on said comparison of the run-time performance scores.