Recursive generation and distribution of command bundles at core array
Some embodiments provide a method for a first processor core of a multi-core arrangement having multiple processor cores. From a queue of command bundles for execution by the processor cores, the method retrieves a first instance of a first command bundle to execute at the first processor core. Multiple cores of the multi-core array also retrieve other instances of the first command bundle to execute. The method executes a first command of the first command bundle instance at the first processor core. Based on a second command of the first command bundle instance, the method adds multiple instances of a second command bundle to the queue for execution by the processor cores.
1 . A method comprising:
at a first worker processor core of a multi-core arrangement, the multi-core arrangement comprising a set of processor cores, wherein one of the processor cores other than the first worker processor core is designated as a leader processor core that executes a scheduler program for executing programs dispatched to the multi-core arrangement:
from a queue of command bundles for execution by the plurality of processor cores, retrieving a first instance of a first command bundle to execute at the first processor core, wherein (i) the leader processor core places a plurality of instances of the first command bundle in the queue, (ii) each instance of the first command bundle specifies a same first set of commands to be performed on different sets of data, and (iii) a plurality of worker processor cores of the multi-core arrangement also retrieve from the queue other instances of the first command bundle to execute;
executing a first command of the first instance of the first command bundle at the first worker processor core; and
based on a second command of the first instance of the first command bundle, adding a plurality of instances of a second command bundle to the queue for execution by the plurality of worker processor cores, each instance of the second command bundle specifying a same second set of commands to be performed on different sets of data,
wherein the worker processor cores that are not designated as the leader processor core (i) are capable of generating pluralities of instances of command bundles and adding the plurality of instances of command bundles to the queue for explicitly parallelizable function calls that specify a number of instances of a specific function to be called and (ii) are not capable of generating pluralities of instances of command bundles for tensor computation commands,
wherein the second command is an explicitly parallelizable function call.
2 . The method of claim 1 , wherein a second worker processor core of the multi-core arrangement that executes a second instance of the first command bundle adds a plurality of instances of a third command bundle to the queue based on a particular command of the second instance of the first command bundle, each instance of the third command bundle specifying a same third set of commands to be performed on different sets of data.
3 . The method of claim 2 , wherein the second worker processor core adds the plurality of instances of the third command bundle to the queue based on a command of the third command bundle that corresponds to the second command of the first command bundle.
4 . The method of claim 1 , wherein retrieving the first instance of the first command bundle comprises:
receiving a notification from the leader processor core that data specifying a plurality of instances of the first command bundle have been placed in the queue by the leader processor core; and
retrieving data specifying the first instance of the first command bundle.
5 . The method of claim 4 , wherein the queue is located in a storage shared with the set of processor cores of the multi-core arrangement.
6 . The method of claim 5 further comprising retrieving data for executing the first instance of the first command bundle from the storage shared with the set of processor cores, wherein the retrieved data is used for executing the first instance of the first command bundle.
7 . The method of claim 6 , wherein other processor cores of the multi-core arrangement that execute respective instances of the first command bundle retrieve other data from the storage for executing the respective instances of the first command bundle.
8 . The method of claim 4 , wherein receiving the notification comprises receiving an assignment of the first instance of the first command bundle to the first worker processor core.
9 . The method of claim 8 , wherein receiving the assignment of the first instance of the first command bundle comprises receiving assignment of a group of instances of the first command bundle, said group including the first instance of the first command bundle.
10 . The method of claim 8 , wherein receiving the assignment of the first instance of the first command bundle comprises receiving a global identifier specifying the first instance of the first command bundle, said global identifier used to retrieve the data used for executing the first instance of the first command bundle.
11 . The method of claim 10 further comprising using the global identifier to read from memory (i) a first memory state common to all of the processor cores of the plurality of worker processor cores and (ii) a second memory state specific to the first worker processor core.
12 . The method of claim 11 , wherein each respective worker processor core in the plurality of worker processor cores that executes a respective instance of the first command bundle reads the first memory state from memory while retrieving data for executing the instance of the particular command bundle assigned to the respective worker processor core.
13 . The method of claim 1 further comprising, at the first worker processor core, after executing the first command bundle instance, notifying the leader processor core of the multi-core arrangement of the completion of the first command bundle instance.
14 . The method of claim 1 , wherein the first command bundle comprises a set of commands, said set of commands comprising at least one scalar command and at least one of (i) a vector command and (ii) a matrix command.
15 . The method of claim 14 , wherein executing the first instance of the first command bundle comprises decoding the commands of the first instance of the first command bundle at a scalar processor of the first worker processor core.
16 . The method of claim 15 , wherein the scalar processor executes the at least one scalar command and distributes (i) any vector instructions to a vector processor of the first worker processor core and (ii) any matrix instructions to a matrix processor of the first worker processor core.
17 . The method of claim 1 further comprising, at the first worker processor core:
executing additional commands of the first instance of the first command bundle until a synchronization point for the second command is reached; and
upon reaching the synchronization point, retrieving an additional command bundle instance from the queue of command bundles and executing commands of the additional command bundle instance.
18 . The method of claim 17 further comprising, after receiving notification that all of the instances of the second command bundle added to the queue based on the second command have been executed, continuing to execute commands of the first instance of the first command bundle.