Scalable monitoring of long running multi-step data intensive workloads
Examples perform monitoring of multiple-step, concurrently executed workflows across distributed nodes. Requests from an intermediate node are classified by a load balancer as monitoring or non-monitoring. Non-monitoring requests are handled by any node; however, monitoring requests are distributed to all nodes via a plurality of queues but handled only by nodes executing the subject workflow. The load balancer receives reports from any node executing the subject workflow, and passes the first report to the intermediate node.
1. A computing system comprising a management node of a plurality of management nodes that, when executed by a processor, monitors multi-step workflows by:
receiving a monitoring request from a load balancer, the monitoring request querying a status of a workflow on the management node, the monitoring request being generated according to one or more of the following: set interval, a policy, in response to an event, and periodically;
comparing an execution identifier (ID) of the received monitoring request to an execution ID of the workflow currently executing on at least one of the plurality of management nodes;
if the compared execution IDs are the same, performing the monitoring request and reporting the performance to the load balancer; and
if the compared execution IDs are not the same, disregarding the monitoring request.
2. The computing system of claim 1 , further comprising:
receiving a non-monitoring request from the load balancer; and
performing the request.
3. The computing system of claim 1 , wherein the execution IDs of current and previously performed workflows are stored by the management node in at least one of a table, database, array, or index.
4. The computing system of claim 1 , wherein the load balancer is other than the plurality of management nodes.
5. The computing system of claim 1 , further comprising a plurality of monitoring queues, wherein each of the monitoring queues is associated with one of the plurality of management nodes.
6. The computing system of claim 1 , wherein receiving the monitoring request comprises receiving a plurality of monitoring requests on-demand from a user, or periodically from an intermediate node.
7. The computing system of claim 1 , wherein receiving the monitoring request comprises receiving the monitoring request from the load balancer via a queue.
8. A method for monitoring multi-step workflows performed by a management node of a plurality of management nodes, the method comprising:
receiving a monitoring request from a load balancer, the monitoring request querying a status of a workflow on the management node, the monitoring request being generated according to one or more of the following: set interval, a policy, in response to an event, and periodically;
comparing an execution identifier (ID) of the received monitoring request to an execution ID of the workflow currently executing on at least one of the plurality of management nodes;
if the compared execution IDs are the same, performing the monitoring request and reporting the performance to the load balancer; and
if the compared execution IDs are not the same, disregarding the monitoring request.
9. The method of claim 8 , further comprising:
receiving a non-monitoring request from the load balancer; and
performing the request.
10. The method of claim 8 , wherein the load balancer is other than the plurality of management nodes.
11. The method of claim 8 , wherein receiving the monitoring request comprises receiving a plurality of monitoring requests on-demand from a user, or periodically from an intermediate node.
12. The method of claim 8 , wherein reporting the performance to the load balancer comprises indicating success or failure of the received monitoring request.
13. The method of claim 8 , wherein the execution IDs of current and previously performed workflows are stored by the management node in at least one of a table, database, array, or index.
14. The method of claim 8 , wherein receiving the monitoring request comprises receiving the monitoring request from the load balancer via a queue.
15. A non-transitory computer-readable medium including instructions that, when executed by a management node of a plurality of management nodes, monitor multi-step workflows, the instructions comprising:
receiving a monitoring request from a load balancer, the monitoring request querying a status of a workflow on the management node, the monitoring request being generated according to one or more of the following: set interval, a policy, in response to an event, and periodically;
comparing an execution identifier (ID) of the received monitoring request to an execution ID of the workflow currently executing on at least one of the plurality of management nodes;
if the compared execution IDs are the same, performing the monitoring request and reporting the performance to the load balancer; and
if the compared execution IDs are not the same, disregarding the monitoring request.
16. The non-transitory computer-readable medium of claim 15 , further comprising:
receiving a non-monitoring request from the load balancer; and
performing the request.
17. The non-transitory computer-readable medium of claim 15 , wherein the execution IDs of current and previously performed workflows are stored by the management node in at least one of a table, database, array, or index.
18. The non-transitory computer-readable medium of claim 15 , wherein receiving the monitoring request comprises receiving a plurality of monitoring requests on-demand from a user, or periodically from an intermediate node.
19. The non-transitory computer-readable medium of claim 15 , wherein receiving the monitoring request comprises receiving the monitoring request from the load balancer via a queue.
20. The non-transitory computer-readable medium of claim 15 , wherein the load balancer is other than the plurality of management.