SYSTEMS AND METHODS FOR PROCESSING USING DIRECTED ACYCLIC GRAPHS

1 . A computer system for constructing and executing a processing graph, the computer system comprising at least one processor in communication with at least one memory device and a graph execution engine, the at least one processor is programmed to:

initiate construction of the processing graph by utilizing initial graph configuration data and at least one root node, wherein the processing graph is constructed to include at least a plurality of nodes including the at least one root node and one or more edges, wherein each edge of the one or more edges connects a pair of nodes, and wherein each node of the processing graph represents a functional component of computation performed by at least one program referenced by that node;

identify a first node of the plurality of nodes within the processing graph;

access a first node definition of the first node from a nodes database, the first node definition identifying a dependency node that represents a functional component of computation generating output data used by the first node;

add a second node to the processing graph as a child of the first node, the second node representing the dependency node;

add a first edge to the processing graph connecting the first node to the second node; and

in response to completing the construction of the processing graph, execute at least one node of the processing graph by recursively traversing the processing graph by starting at the at least one root node and visiting each of the plurality of nodes, wherein traversing the processing graph comprises, during execution of a parent node of the at least one node, processing the at least one node by receiving input data from the parent node or output data from one or more child nodes of that at least one node, and wherein visiting each node comprises (i) causing the graph execution engine to execute one or more referenced programs of that visited node and (ii) for each child node of that visited node, causing the graph execution engine to traverse each child node for further execution.

2 . The computer system of claim 1 , wherein the processing graph is a directed acyclic graph, wherein the first edge is directed from the first node to the second node.

3 . The computer system of claim 1 , wherein each node of the plurality of nodes includes a node unique ID (UID), wherein the first node definition identifies the dependency node based upon a node UID of that dependency node.

4 . The computer system of claim 1 , wherein the first node definition includes at least one output field that identifies an output generated by the first node during execution of the first node.

5 . The computer system of claim 4 , wherein the output generated by the first node during execution of the first node is passed to a parent node of the first node after execution of the first node is complete.

6 . The computer system of claim 1 , wherein the first node definition includes at least one input field that identifies an input variable accepted by the first node during execution of the first node.

7 . The computer system of claim 6 , wherein the input variable is provided by a parent node of the first node prior to initiating execution of the first node.

8 . The computer system of claim 1 , wherein the first node definition further includes a first program that is executed by the computer system during execution of the first node.

9 . The computer system of claim 8 , wherein the first program includes source code written in an embedded programming language provided on the computer system.

10 . The computer system of claim 8 , wherein the first program includes a reference to a local program stored on a storage device local to the computer system, and wherein the local program is one or more of a reference to a script file, an executable binary file, a local library of functions, and a local service.

11 . (canceled)

12 . The computer system of claim 8 , wherein the first program includes a reference to an external program performed by another computer system.

13 . The computer system of claim 12 , wherein the first program is one or more of a reference to a cloud service, an application programming interface (API) service, a third party service, and a network-based service.

14 . The computer system of claim 1 , wherein the at least one processor is further configured to complete the processing graph by recursively traversing the processing graph to identify all dependency nodes that are not yet included in the processing graph and add all of the dependency nodes to the processing graph.

15 . The computer system of claim 1 , wherein executing the processing graph further includes traversing the processing graph until each node in the processing graph has been executed.

16 . A computer-implemented method for constructing and executing a processing graph, the method implemented by a computer device including at least one processor in communication with at least one memory device and a graph execution engine, the method comprising:

initiating construction of the processing graph by utilizing initial graph configuration data and at least one root node, wherein the processing graph is constructed to include at least a plurality of nodes including the at least one root node and one or more edges, wherein each edge of the one or more edges connects a pair of nodes, and wherein each node of the processing graph represents a functional component of computation performed by at least one program referenced by that node;

identifying a first node of the plurality of nodes within the processing graph;

accessing a first node definition of the first node from a nodes database, the first node definition identifying a dependency node that represents a functional component of computation generating output data used by the first node;

adding a second node to the processing graph as a child of the first node, the second node representing the dependency node;

adding a first edge to the processing graph connecting the first node to the second node; and

in response to completing the construction of the processing graph, executing at least one node of the processing graph by recursively traversing the processing graph by starting at the at least one root node and visiting each of the plurality of nodes, wherein traversing the processing graph comprises, during execution of a parent node of the at least one node, processing the at least one node by receiving input data from the parent node or output data from one or more child nodes of that at least one node, and wherein visiting each node includes (i) causing the graph execution engine to execute one or more referenced programs of that visited node and (ii) for each child node of that visited node, causing the graph execution engine to traverse each child node for further execution.

17 . The method of claim 16 , wherein the processing graph is a directed acyclic graph, wherein the first edge is directed from the first node to the second node.

18 . The method of claim 16 , wherein each node of the plurality of nodes includes a node unique ID (UID), wherein the first node definition identifies the dependency node based upon a node UID of that dependency node.

19 . The method of claim 16 , wherein the first node definition includes at least one output field that identifies an output generated by the first node during execution of the first node.

20 . The method of claim 19 , wherein the output generated by the first node during execution of the first node is passed to a parent node of the first node after execution of the first node is complete.

21 . The computer system of claim 1 , wherein the at least one processor is further programmed to:

identify one or more groups of independent nodes of the plurality of nodes; and

simultaneously processing the one or more groups of independent nodes.