DATA FLOW GRAPH COMPUTATION FOR MACHINE LEARNING
Techniques are disclosed for data manipulation that enables data flow graph computation for machine learning. A plurality of processing elements within a reconfigurable fabric is configured to implement a data flow graph. The nodes of the data flow graph include variable notes. The plurality of processing elements is initialized with a plurality of process agents. A first set of buffers is initialized for a first process agent, where the first process agent corresponds to a starting node of the data flow graph. A fire signal is issued for the starting node based on the first set of buffers being initialized. Results of operations are collected by a further process agent following receipt of the fire signal. The data flow graph computation can be paused by loading invalid data or by withholding new data from entering the data flow graph. The pausing can be controlled by an execution manager.
1 . A processor-implemented method for data manipulation comprising:
configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph;
loading the plurality of processing elements with a plurality of process agents;
initializing a first set of buffers for a first process agent, wherein the first process agent corresponds to a starting node of the data flow graph; and
issuing a fire signal for the starting node, based on the first set of buffers being initialized.
2 . The method of claim 1 further comprising collecting results of operations by a further process agent following receipt of the fire signal.
3 . The method of claim 1 further comprising providing a done signal from the first process agent when data from the first set of buffers has been consumed by the first process agent.
4 . The method of claim 1 wherein a set of nodes within the data flow graph comprise variable nodes.
5 . (canceled)
6 . The method of claim 1 further comprising initializing a second set of buffers for a second process agent, wherein the second process agent corresponds to a successor node of the data flow graph.
7 . The method of claim 6 further comprising issuing a fire signal for the successor node, based on the first process agent completing issuing a done signal.
8 . The method of claim 7 wherein the done signal indicates that valid data is present in the second set of buffers.
9 . The method of claim 1 wherein the first set of buffers is initialized with valid data.
10 - 11 . (canceled)
12 . The method of claim 1 wherein the reconfiguring and the loading are controlled by a session manager.
13 . The method of claim 1 wherein the initializing and the issuing are controlled by an execution manager.
14 . The method of claim 1 further comprising pausing the data flow graph.
15 . The method of claim 14 wherein the pausing is accomplished by loading invalid data.
16 . The method of claim 15 wherein the pausing is controlled by an execution manager.
17 . The method of claim 14 wherein the pausing is accomplished by withholding new data from entering the data flow graph.
18 . The method of claim 17 wherein the pausing is controlled by an execution manager.
19 . The method of claim 14 further comprising restarting a paused data flow graph.
20 . The method of claim 19 wherein the restarting includes issuing a run command to each node within the data flow graph.
21 . The method of claim 19 wherein the restarting includes providing new data to the starting node.
22 . The method of claim 14 further comprising reading a status of each node within the data flow graph.
23 - 24 . (canceled)
25 . The method of claim 1 wherein the processing elements are controlled by circular buffers.
26 . The method of claim 25 wherein the circular buffers are statically scheduled.
27 - 29 . (canceled)
30 . A computer program product embodied in a non-transitory computer readable medium for data manipulation, the computer program product comprising code which causes one or more processors to perform operations of:
configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph;
loading the plurality of processing elements with a plurality of process agents;
initializing a first set of buffers for a first process agent, wherein the first process agent corresponds to a starting node of the data flow graph; and
issuing a fire signal for the starting node, based on the first set of buffers being initialized.
31 . A computer system for data manipulation comprising:
a memory which stores instructions;
one or more processors attached to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:
configure a plurality of processing elements within a reconfigurable fabric to implement a data flow graph;
load the plurality of processing elements with a plurality of process agents;
initialize a first set of buffers for a first process agent, wherein the first process agent corresponds to a starting node of the data flow graph; and
issue a fire signal for the starting node, based on the first set of buffers being initialized.