Service quality evaluator having adaptive evaluation criteria
View Patent ↗An improved method is provided for evaluating service quality of a software application residing in a computing environment. The method includes: monitoring first and second metrics indicative of performance of the software application; evaluating the first metric in relation to a first evaluation criterion; evaluating the second metric in relation to a second evaluation criterion different than the first evaluation criterion; and determining the state of service for the software application based in part on the evaluation of the second metric. Over time, the second evaluation criterion is adapted based on the outcome of the evaluation of the first metric.
1. A method for evaluating service quality of a software application residing in a computing environment, comprising: determining a real response time for the software application derived when the software application is processing transactions initiated by end users of the software application; determining a synthetic response time for the software application derived when the software application is processing transactions initiated by a script; monitoring a weighting metric indicative of performance of the software application; evaluating the weighting metric in relation to an evaluation criteria using a service quality evaluator executing on a computer processor; quantifying, by the service quality evaluator, a first state of service for the software application depending on the evaluation of the weighing metric, where the first state of service is quantified in accordance with the synthetic response time for the software application when the weighting metric matches the evaluation criteria and the first state of service is quantified in accordance with the real response time for the software application when the weighting metric does not match the evaluation criteria; determining a current time of day; evaluating a second state of service for the software application in accordance with a first evaluation directive using the service quality evaluator; assigning a first weighting factor to the second state of service by the service quality evaluator when the current time of day falls within a schedule of service for the software application; assigning a second weighting factor to the second state of service by the service quality evaluator when the current time of day falls outside the schedule of service for the software application; evaluating a third state of service for the software application in accordance with a second evaluation directive different than the first evaluation directive; assigning the second weighting factor to the third state of service determined in accordance with the second evaluation directive when the current time of day falls outside the schedule of service for the software application; assigning the first weighting factor to the third state of service determined in accordance with the second evaluation directive when the current time of day falls within the schedule of service for the software application; and aggregating the second state of service determined in accordance with the first evaluation directive with the third state of service determined in accordance with the second evaluation directive.
2. The method of claim 1 further comprises determining a current time of day; evaluating the first state of service primarily in accordance the real response time when the current time of day falls within a schedule of service for the software application; and evaluating the first state of service primarily in accordance with the synthetic response time when the current time of data falls outside the schedule of service for the software application.
3. The method of claim 1 further comprises: determining a number of current users of the software application; evaluating the first state of service primarily in accordance the real response time when the number of current users exceeds a threshold; and evaluating the first state of service primarily in accordance with the synthetic response time when the number of current users is less than the threshold.
4. A method for evaluating service quality of a software application residing in a computing environment, comprising: determining a real response time for the software application derived when the software application is processing transactions initiated by end users of the software application; determining a synthetic response time for the software application derived when the software application is processing transactions initiated by a script; monitoring a weighting metric indicative of performance of the software application; evaluating the weighting metric in relation to an evaluation criteria using a service quality evaluator executing on a computer processor; and quantifying, by the service quality evaluator, a first state of service for the software application depending upon on the evaluation of the weighting metric, where the first state of service is quantified in accordance with the synthetic response time for the software application when the weighting metric matches the evaluation criteria and the first state of service is quantified in accordance with the real response time for the software application when the weighting metric does not match the evaluation criteria; determining a current number of users of the software application; evaluating a second state of service for the software application in accordance with a first evaluation directive using the service quality evaluator; assigning a first weighting factor to the second state of service by the service quality evaluator when the current number of users exceeds a threshold; assigning a second weighting factor to the second state of service by the service quality evaluator when the current number of users is less than the threshold; evaluating a third state of service for the software application in accordance with a second evaluation directive different than the first evaluation directive; assigning the second weighting factor to the third state of service determined in accordance with the second evaluation directive when the current number of users is less than the threshold; assigning the first weighting factor to the third state of service determined in accordance with the second evaluation directive when the current number of users exceeds the threshold; and aggregating the second state of service determined in accordance with the first evaluation directive with the third state of service determined in accordance with the second evaluation directive.
5. A method for evaluating service quality of a software application residing in a computing environment, comprising: determining a real response time for the software application derived when the software application is processing transactions initiated by end users of the software application; determining a synthetic response time for the software application derived when the software application is processing transactions initiated by a script; determining a number of current users of the software application; quantifying, by a service quality evaluator, the first state of service as a function of the real response time when the number of current users exceeds a threshold, where the service quality evaluator is executing on a computer processor; and quantifying, by the service quality evaluator, the first state of service as a function of the synthetic response time when the number of current users is less than the threshold; determining, by the service quality evaluator, a first numeric value indicating a second state of service for the software application based on the real response time; selecting, by the service quality evaluator, a first weighting factor as a function of the number of current users; multiplying, by the service quality evaluator, the first numeric value by the first weighting factor to generate a weighted first numeric value; determining, by the service quality evaluator, a second numeric value indicating a third state of service for the software application based on the synthetic response time; selecting, by the service quality evaluator, a second weighting factor as a function of the number of current users; multiplying, by the service quality evaluator, the second numeric value by the second weighting factor to generate a weighted second numeric value; and assigning a fourth state of service for the software application by aggregating the weighted first numeric value with the weighted second numeric value.
6. A method for evaluating service quality of a software application residing in a computing environment, comprising: determining a percentage of slow transactions for the software application derived when the software application is processing transactions initiated by end users of the software application; determining a synthetic response time for the software application derived when the software application is processing transactions initiated by a script; determining a number of current users of the software application; quantifying, by a service quality evaluator, a state of service for the software application as a function of the percentage of slow transactions when the number of current users exceeds a threshold, where the service quality evaluator is executing on a computer processor; and quantifying, by the service quality evaluator, the state of service as a function of the synthetic response time when the number of current users is less than the threshold; determining, by the service quality evaluator, a first numeric value indicating a second state of service for the software application based on the percentage of slow transactions; selecting, by the service quality evaluator, a first weighting factor as a function of the number of current users; multiplying, by the service quality evaluator, the first numeric value by the first weighting factor to generate a weighted first numeric value; determining, by the service quality evaluator, a second numeric value indicating a third state of service for the software application based on the synthetic response time; selecting, by the service quality evaluator, a second weighting factor as a function of the number of current users; multiplying, by the service quality evaluator, the second numeric value by the second weighting factor to generate a weighted second numeric value; and assigning a fourth state of service for the software application by aggregating the weighted first numeric value with the weighted second numeric value.