Monitoring performance deviations
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for monitoring performance deviations. In some implementations, a workflow that is configured to monitor performance of an operation of one or more computers is run. In running the workflow, a performance measure for the operation is determined. In running the workflow, a reference performance level for the operation is determined. In running the workflow, it is determined whether the performance measure indicates a deviation from the reference performance level for the operation. In running the workflow, an action specified by the workflow is selectively performed based on whether the performance measure is determined to indicate a deviation from the reference performance level for the operation.
1. A method performed by one or more computers, the method comprising:
receiving, by the one or more computers, a data package over a communication network, the data package specifying a workflow to monitor performance of a particular type of operation, wherein the data package specifies criteria to determine, for a system that runs the workflow, a customized performance baseline based on a plurality of historical performance measures for the system that runs the workflow, and wherein the data package specifies multiple different management actions to improve performance in response to different levels of difference in performance with respect to the customized performance baseline; and
in response to receiving the data package over the communication network, running, by the one or more computers, the workflow specified by the data package to monitor performance of an operation of the particular type by of the one or more computers, wherein running the workflow comprises:
initiating the operation of the particular type;
determining a performance measure for the operation initiated as part of the workflow;
determining a customized performance baseline for the operation for the one or more computers based on the criteria specified in the data package, wherein the criteria specifies that the customized performance baseline is determined from previous performance measures for each of multiple instances of the one or more computers performing the operation;
determining a level of difference between the performance measure for the operation and the customized performance baseline for the operation;
selecting a management action, from among the multiple different management actions that the data package specifies, that corresponds to the determined level of difference in performance with respect to the customized performance baseline; and
performing, by the one or more computers, the selected management action corresponding to the determined level of difference in performance to increase the performance of the operation for the one or more computers.
2. The method of claim 1 , wherein the performance measure is specified by the data package and is indicative of latency, response time, task completion time, transmission time, data rate, bandwidth usage, capacity, reliability, accuracy, efficiency, availability, power usage, processing speed, throughput, or level of concurrency.
3. The method of claim 1 , comprising repeatedly performing the workflow to monitor performance of multiple instances of the operation at different times.
4. The method of claim 3 , comprising periodically initiating the operation based on the data package and monitoring the performance of the periodic instances of the operation.
5. The method of claim 1 , wherein the workflow is configured to monitor an operation involving a specific data object, data set, document, application, service, or user.
6. The method of claim 1 , wherein the workflow specifies an action to be taken in response to detecting that the performance measure indicates performance that is higher than the customized performance baseline by at least a minimum amount.
7. The method of claim 1 , wherein the workflow specifies an action to be taken in response to detecting that the performance measure indicates performance that is less than the customized performance baseline by at least a minimum amount.
8. The method of claim 1 , wherein the customized performance baseline is determined based on a previous performance of the one or more computers during a sliding window of time.
9. The method of claim 1 , wherein the operation comprises:
loading a document;
creating a data cube;
responding to a query;
retrieving a file;
providing data for a dashboard; or
generating a report or visualization.
10. The method of claim 1 , wherein the data package for the workflow specifies that the selected management action is conditional, to be performed when a particular condition or context is present in addition to a level of performance difference from the customized performance baseline; and
wherein performing the selected management action is further based on determining that the particular condition or context is present for the one or more computers.
11. The method of claim 1 , wherein the data package specifies different configuration settings to apply in response to detection of different levels of difference in performance with respect to the customized performance baseline; and
wherein performing the selected management action comprises applying a configuration setting selected from among the different configuration settings based on the determined level of difference with respect to the customized performance baseline.
12. The method of claim 1 , wherein the selected management action comprises restarting a computing environment.
13. The method of claim 1 , wherein the selected management action comprises changing a distribution of tasks or traffic among multiple computing systems.
14. The method of claim 1 , wherein the selected management action comprises starting one or more new instances of a server environment in a cloud computing system.
15. The method of claim 1 , wherein the selected management action comprises checking for, downloading, or initiating a software update for the one or more computers.
16. The method of claim 1 , wherein the selected management action comprises increasing an allocation of additional computing resources to the one or more computers.
17. A system comprising:
one or more computers; and
one or more computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
receiving, by the one or more computers, a data package over a communication network, the data package specifying a workflow to monitor performance of a particular type of operation, wherein the data package specifies criteria to determine, for a system that runs the workflow, a customized performance baseline based on a plurality of historical performance measures for the system that runs the workflow, and wherein the data package specifies multiple different management actions to improve performance in response to different levels of difference in performance with respect to the customized performance baseline; and
in response to receiving the data package over the communication network, running, by the one or more computers, the workflow specified by the data package to monitor performance of an operation of the particular type by of the one or more computers, wherein running the workflow comprises:
initiating the operation of the particular type;
determining a performance measure for the operation initiated as part of the workflow;
determining a customized performance baseline for the operation for the one or more computers based on the criteria specified in the data package, wherein the criteria specifies that the customized performance baseline is determined from previous performance measures for each of multiple instances of the one or more computers performing the operation;
determining a level of difference between the performance measure for the operation and the customized performance baseline for the operation;
selecting a management action, from among the multiple different management actions that the data package specifies, that corresponds to the determined level of difference in performance with respect to the customized performance baseline; and
performing, by the one or more computers, the selected management action corresponding to the determined level of difference in performance to increase the performance of the operation for the one or more computers.
18. The system of claim 17 , wherein the performance measure is indicative of latency, response time, task completion time, transmission time, data rate, bandwidth usage, capacity, reliability, accuracy, efficiency, availability, power usage, processing speed, throughput, or level of concurrency.
19. One or more non-transitory computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
receiving, by the one or more computers, a data package over a communication network, the data package specifying a workflow to monitor performance of a particular type of operation, wherein the data package specifies criteria to determine, for a system that runs the workflow, a customized performance baseline based on a plurality of historical performance measures for the system that runs the workflow, and wherein the data package specifies multiple different management actions to improve performance in response to different levels of difference in performance with respect to the customized performance baseline; and
in response to receiving the data package over the communication network, running, by the one or more computers, the workflow specified by the data package to monitor performance of an operation of the particular type by of the one or more computers, wherein running the workflow comprises:
initiating the operation of the particular type;
determining a performance measure for the operation initiated as part of the workflow;
determining a customized performance baseline for the operation for the one or more computers based on the criteria specified in the data package, wherein the criteria specifies that the customized performance baseline is determined from previous performance measures for each of multiple instances of the one or more computers performing the operation;
determining a level of difference between the performance measure for the operation and the customized performance baseline for the operation;
selecting a management action, from among the multiple different management actions that the data package specifies, that corresponds to the determined level of difference in performance with respect to the customized performance baseline; and
performing, by the one or more computers, the selected management action corresponding to the determined level of difference in performance to increase the performance of the operation for the one or more computers.