Computation graph compiling method and apparatus

1 . A data processing apparatus, comprising:

graph partitioning circuitry configured to receive a computation graph, the computation graph comprising a plurality of nodes representing a plurality of operators and a plurality of edges representing relationships amongst the plurality of operators, and to divide the computation graph into a plurality of partitions, each partition comprising one or more nodes and/or edges;

a graph compiler comprising graph compilation circuitry configured to compile a computation graph to generate one or more compilation outputs;

storage to store the one or more compilation outputs; and

execution circuitry configured to execute the one or more compilation outputs;

wherein the graph compilation circuitry is configured to:

compile a first partition of the plurality of partitions to generate a first compilation output; and

output the first compilation output to a first target portion of the storage assigned to the first partition; and

wherein the execution circuitry is configured to:

collect performance statistics with respect to one or more of the plurality of partitions and/or a compilation output of the one or more of the plurality of partitions, and to feedback the collected performance statistics to the graph partitioning circuitry and/or the graph compilation circuitry.

2 . The apparatus of claim 1 , wherein the graph compilation circuitry is configured to compile a second partition of the plurality of partitions to generate a second compilation output in parallel with the first partition.

3 . The apparatus of claim 1 , wherein the graph compilation circuitry compiles a partition of the plurality of partitions by applying a scheduling algorithm to the partition to generate a schedule, wherein the scheduling algorithm is configured to determine one or more data processing operations based on the one or more nodes and/or edges and to assign the one or more data processing operations to the execution circuitry in the schedule.

4 . The apparatus of claim 1 , wherein the graph partitioning circuitry divides the computation graph into a plurality of partitions by applying a partitioning algorithm, wherein the partitioning algorithm is configured to divide the computation graph into a plurality of partitions based on one or more partition objectives and assign a target portion of the storage to at least one of the plurality of partitions.

5 . The apparatus of claim 4 , wherein the one or more partition objectives comprise dividing the computation graph into a plurality of partitions based on optimizing the target portion of the storage with respect to a size of a compilation output of the at least one of the plurality of partitions assigned thereto.

6 . The apparatus of claim 4 , wherein the one or more partition objectives comprise dividing the computation graph into a plurality of linear partitions each comprising only nodes of a linear topology.

7 . The apparatus of claim 4 , wherein the one or more partition objectives comprise dividing the computation graph into a plurality of same size and/or same shape partitions.

8 . The apparatus of claim 1 , wherein the execution circuitry is configured to execute the first compilation output upon the first target portion of the storage receiving the first compilation output.

9 . The apparatus of claim 8 , wherein the graph compilation circuitry is configured to compile a second partition of the plurality of partitions to generate a second compilation output while the execution circuitry executes the first compilation output, and to output the second compilation output to a second target portion of the storage assigned to the second partition.

10 . The apparatus of claim 1 , wherein the performance statistics comprises, with respect to one or more of the plurality of partitions and/or a compilation output of the one or more of the plurality of partitions, a compiling time, an execution time, an inference time, or a combination thereof.

11 . The apparatus of claim 1 , wherein:

the graph partitioning circuitry is configured to adjust division of the computation graph into a plurality of partitions based on the collected performance statistics to reduce a compiling time, an execution time, an inference time, or a combination thereof; and/or

the graph compilation circuitry is configured to adjust compilation of a partition of the plurality of partitions to generate a compilation output based on the collected performance statistics to reduce a compiling time, an execution time, an inference time, or a combination thereof.

12 . The apparatus of claim 11 , wherein the graph partitioning circuitry and/or the graph compilation circuitry is configured to terminate the adjustment of division of the computation graph into the plurality of partitions when a predetermined elapsed time is reached, when a predetermined proportion of search space is covered, and/or when a predetermined performance threshold is reached for one or more of the plurality of partitions and/or a compilation output of the one or more of the plurality of partitions.

13 . A method comprising:

receiving a computation graph, the computation graph comprising a plurality of nodes representing a plurality of operators and a plurality of edges representing relationships amongst the plurality of operators;

dividing the computation graph into a plurality of partitions, each partition comprising one or more nodes and/or edges;

compiling, at a graph compiler, a computation graph to generate one or more compilation outputs;

storing the one or more compilation outputs in storage; and

executing the one or more compilation outputs,

wherein compiling a computation graph comprises:

compiling a first partition of the plurality of partitions to generate a first compilation output; and

outputting the first compilation output to a first target portion of the storage assigned to the first partition; and

wherein executing the one or more compilation outputs comprises:

collecting performance statistics with respect to one or more of the plurality of partitions and/or a compilation output of the one or more of the plurality of partitions, and to feedback the collected performance statistics to the dividing the computation graph and/or the computation graph compiling.

14 . The method of claim 13 , further comprising compiling a second partition of the plurality of partitions to generate a second compilation output in parallel with the first partition.

15 . The method of claim 13 , wherein compiling a partition of the plurality of partitions comprises applying a scheduling algorithm to the partition to generate a schedule, wherein the scheduling algorithm is configured to determine one or more data processing operations based on the one or more nodes and/or edges and to assign the one or more data processing operations to execution circuitry in the schedule.

16 . The method of claim 13 , wherein dividing the computation graph into a plurality of partitions comprises applying a partitioning algorithm, wherein the partitioning algorithm is configured to divide the computation graph into a plurality of partitions based on one or more partition objectives and assign a target portion of the storage to at least one of the plurality of partitions.

17 . The method of claim 16 , wherein the one or more partition objectives comprise dividing the computation graph into a plurality of partitions based on optimizing the target portion of the storage with respect to a size of a compilation output of the at least one of the plurality of partitions assigned thereto.

18 . A non-transitory data carrier carrying code which, when implemented on a processor, causes the processor to carry out the method of claim 13 .

19 . A non-transitory computer-readable medium comprising instructions for controlling a host data processing apparatus to provide an instruction execution environment comprising:

graph partitioning program logic configured to receive a computation graph, the computation graph comprising a plurality of nodes representing a plurality of operators and a plurality of edges representing relationships amongst the plurality of operators, and to divide the computation graph into a plurality of partitions, each partition comprising one or more nodes and/or edges;

a graph compiler comprising graph compilation program logic configured to compile a computation graph to generate one or more compilation outputs;

storage to store the one or more compilation outputs; and

execution circuitry configured to execute the one or more compilation outputs;

wherein the graph compilation program logic is configured to:

compile a first partition of the plurality of partitions to generate a first compilation output; and

output the first compilation output to a first target portion of the storage assigned to the first partition; and

wherein the execution circuitry is configured to:

collect performance statistics with respect to one or more of the plurality of partitions and/or a compilation output of the one or more of the plurality of partitions, and to feedback the collected performance statistics to the graph partitioning program logic and/or the graph compilation program logic.