FACILITATING PERFORMANCE MONITORING FOR PERIODICALLY SCHEDULED WORKFLOWS

1 . A computer-implemented method for monitoring a workflow, the method comprising:

monitoring an execution time for the workflow, wherein the workflow comprises a set of jobs that execute on nodes of a computing cluster;

while monitoring the execution time for the workflow, monitoring execution times for individual jobs in the set of jobs that comprise the workflow;

determining an execution-time threshold for the workflow based on prior executions of the workflow;

if a monitored execution time for the workflow exceeds the determined execution-time threshold for the workflow, sending an alert about the workflow to a user; and

enabling the user to examine the monitored execution time for the workflow and the monitored execution times for the individual jobs that comprise the workflow.

2 . The computer-implemented method of claim 1 , wherein the method further comprises:

determining execution-time thresholds for jobs that comprise the workflow based on previous executions of the jobs; and

if an execution time for a job exceeds the determined execution-time threshold for the job, sending an alert about the job to the user.

3 . The computer-implemented method of claim 1 , wherein the method further comprises enabling the user to examine a dependency graph for the workflow to facilitate determining a solution to a performance problem for the workflow, wherein the dependency graph specifies dependencies between jobs in the workflow, and wherein a dependency between a first job and a second job indicates that the first job must complete before the second job can begin executing.

4 . The computer-implemented method of claim 1 , wherein determining the execution-time threshold for the workflow includes:

determining a mean value and a standard deviation for the execution time for the workflow based on prior successful executions of the workflow; and

adding the determined standard deviation and a buffer time to the determined mean value to produce the execution-time threshold.

5 . The computer-implemented method of claim 4 , wherein enabling the user to examine the monitored execution time for the workflow involves enabling the user to examine parameters for the workflow, including:

an identifier for the workflow;

a day-of-the-week that the workflow was executed on;

a start time for the workflow;

an end time for the workflow;

a run time for the workflow;

an execution status for the workflow;

a mean value for the execution time for the workflow;

a standard deviation for the execution time for the workflow; and

the execution-time threshold for the workflow.

6 . The computer-implemented method of claim 4 , further comprising enabling the user to configure:

the buffer time; and

a magnitude for the standard deviation.

7 . The computer-implemented method of claim 1 ,

wherein monitoring the execution time for the workflow involves monitoring values for one or more internal counters for events associated with the workflow; and

wherein enabling the user to examine the monitored execution time for the workflow also includes enabling the user to examine the monitored values for the one or more internal counters.

8 . A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for monitoring a workflow, the method comprising:

monitoring an execution time for the workflow, wherein the workflow comprises a set of jobs that execute on nodes of a computing cluster;

while monitoring the execution time for the workflow, monitoring execution times for individual jobs in the set of jobs that comprise the workflow;

determining an execution-time threshold for the workflow based on prior executions of the workflow;

if a monitored execution time for the workflow exceeds the determined execution-time threshold for the workflow, sending an alert about the workflow to a user; and

enabling the user to examine the monitored execution time for the workflow and the monitored execution times for the individual jobs that comprise the workflow.

9 . The non-transitory computer-readable storage medium of claim 8 , wherein the method further comprises:

determining execution-time thresholds for jobs that comprise the workflow based on previous executions of the jobs; and

if an execution time for a job exceeds the determined execution-time threshold for the job, sending an alert about the job to the user.

10 . The non-transitory computer-readable storage medium of claim 8 , wherein the method further comprises enabling the user to examine a dependency graph for the workflow to facilitate determining a solution to a performance problem for the workflow, wherein the dependency graph specifies dependencies between jobs in the workflow, and wherein a dependency between a first job and a second job indicates that the first job must complete before the second job can begin executing.

11 . The non-transitory computer-readable storage medium of claim 8 , wherein determining the execution-time threshold for the workflow includes:

determining a mean value and a standard deviation for the execution time for the workflow based on prior successful executions of the workflow; and

adding the determined standard deviation and a buffer time to the determined mean value to produce the execution-time threshold.

12 . The non-transitory computer-readable storage medium of claim 11 , wherein enabling the user to examine the monitored execution time for the workflow involves enabling the user to examine parameters for the workflow, including:

an identifier for the workflow;

a day-of-the-week that the workflow was executed on;

a start time for the workflow;

an end time for the workflow;

a run time for the workflow;

an execution status for the workflow;

a mean value for the execution time for the workflow;

a standard deviation for the execution time for the workflow; and

the execution-time threshold for the workflow.

13 . The non-transitory computer-readable storage medium of claim 11 , further comprising enabling the user to configure:

the buffer time; and

a magnitude for the standard deviation.

14 . The non-transitory computer-readable storage medium of claim 8 ,

wherein monitoring the execution time for the workflow involves monitoring values for one or more internal counters for events associated with the workflow; and

wherein enabling the user to examine the monitored execution time for the workflow also includes enabling the user to examine the monitored values for the one or more internal counters.

15 . A system that monitors execution of a workflow, comprising:

a computing cluster comprising a plurality of processors and associated memories;

a monitoring mechanism that executes on the computing cluster and is configured to,

monitor an execution time for the workflow, wherein the workflow comprises a set of jobs that execute on nodes of a computing cluster;

monitor execution times for individual jobs in the set of jobs that comprise the workflow;

determine an execution-time threshold for the workflow based on prior executions of the workflow;

if a monitored execution time for the workflow exceeds the determined execution-time threshold for the workflow, send an alert about the workflow to a user; and

enable the user to examine the monitored execution time for the workflow and the monitored execution times for the individual jobs that comprise the workflow.

16 . The system of claim 15 , wherein the monitoring mechanism is further configured to:

determine execution-time thresholds for jobs that comprise the workflow based on previous executions of the jobs; and

if an execution time for a job exceeds the determined execution-time threshold for the job, send an alert about the job to the user.

17 . The system of claim 15 , wherein the monitoring mechanism is further configured to enable the user to examine a dependency graph for the workflow to facilitate determining a solution to a performance problem for the workflow, wherein the dependency graph specifies dependencies between jobs in the workflow, and wherein a dependency between a first job and a second job indicates that the first job must complete before the second job can begin executing.

18 . The system of claim 15 , wherein while determining the execution-time threshold for the workflow, the monitoring mechanism is configured to:

determine a mean value and a standard deviation for the execution time for the workflow based on prior successful executions of the workflow; and

add the determined standard deviation and a buffer time to the determined mean value to produce the execution-time threshold.

19 . The system of claim 18 , wherein enabling the user to examine the monitored execution time for the workflow involves enabling the user to examine parameters for the workflow, including:

an identifier for the workflow;

a day-of-the-week that the workflow was executed on;

a start time for the workflow;

an end time for the workflow;

a run time for the workflow;

an execution status for the workflow;

a mean value for the execution time for the workflow;

a standard deviation for the execution time for the workflow; and

the execution-time threshold for the workflow.

20 . The system of claim 18 , wherein the monitoring mechanism is further configured to enable the user to set:

the buffer time; and

a magnitude for the standard deviation.

21 . The system of claim 15 ,

wherein while monitoring the execution time for the workflow, the monitoring mechanism is configured to monitor values for one or more internal counters for events associated with the workflow; and

wherein while enabling the user to examine the monitored execution time for the workflow, the monitoring mechanism is configured to enable the user to examine the monitored values for the one or more internal counters.