PROCESSOR SYSTEM AND METHOD FOR INCREASING DATA-TRANSFER BANDWIDTH DURING EXECUTION OF A SCHEDULED PARALLEL PROCESS

1 . A method for executing a neural network by a processor system comprising a shared memory unit, a set of primary memory units, and a set of processing units, the method comprising:

storing, in the shared memory unit:

a first weight tensor at a first source address, the first weight tensor including a set of weight tensor partitions; and

a first input tensor at a second source address, the first input tensor larger than the first weight tensor;

broadcasting the first input tensor from the second source address to a first relative destination address in the set of primary memory units;

for each processing unit in the set of processing units:

transferring a weight tensor partition in the set of weight tensor partitions from the first source address to a first destination address in the primary memory unit of the processing unit; and

at the processing unit, generating an output tensor partition of a first output tensor based on the first input tensor and the weight tensor partition;

storing, in the shared memory unit:

a second weight tensor at a third source address; and

a second input tensor at a fourth source address, the second input tensor:

comprising a set of input tensor partitions; and

smaller than the second weight tensor;

broadcasting the second weight tensor from the third source address to a second relative destination address in the set of primary memory units; and

for each processing unit in the set of processing units:

transferring an input tensor partition in the set of input tensor partitions from the fourth source address to a second destination address in the primary memory unit of the processing unit; and

at each processing unit in the set of processing units, generating an output tensor partition of a second output tensor based on the second weight tensor and the input tensor partition.

2 . The method of claim 1 :

wherein a first layer of the neural network comprises the first input tensor and the first weight tensor; and

wherein a second layer of the neural network comprises the second input tensor and the second weight tensor.

3 . The method of claim 2 , further comprising, at a scheduling application:

scheduling the neural network to define a set of input-broadcast layers comprising the first layer; and

scheduling the neural network to define a set of weight-broadcast layer comprising the second layer.

4 . The method of claim 3 , further comprising at the scheduling application:

for each layer of the neural network, calculating a heuristic based on the relative size of an input tensor of the layer and a weight tensor of the layer; and

designating the layer as one of an input-broadcast layer or a weight-broadcast layer.

5 . The method of claim [ 0012 ]:

wherein broadcasting the first input tensor from the second source address to the first relative destination address comprises:

via a direct memory access core of the processor system, issuing a first read request for the first input tensor at the second source address;

in response to the first read request, loading the first input tensor into a data buffer;

via the direct memory access core, issuing a first write request specifying the first relative destination address; and

via a set of broadcast buses of the processor system, transferring the first input tensor from the data buffer to the first relative destination address in each primary memory unit in the set of primary memory units; and

wherein broadcasting the second weight tensor from the third source address to the second relative destination address comprises:

via the direct memory access core of the processor system, issuing a second read request for the second weight tensor at the third source address;

in response to the second read request, loading the second weight tensor into the data buffer; and

via the direct memory access core, issuing a second write request specifying the second relative destination address; and

via the set of broadcast buses, transferring the second weight tensor from the data buffer to the second relative destination address in each primary memory unit in the set of primary memory units.

6 . The method of claim 5 :

wherein transferring the first input tensor from the data buffer to the first relative destination address comprises, at each broadcast bus in the set of broadcast buses:

selecting a multicast mode for the broadcast bus;

transferring the first input tensor from the data buffer to the first relative destination address in each primary memory unit in a subset of primary memory units corresponding to the broadcast bus, the broadcast bus operating in multicast mode.

transferring the second input tensor from the data buffer to the second relative destination address comprises, at each broadcast bus in the set of broadcast buses:

selecting the multicast mode for the broadcast bus; and

transferring the second weight tensor from the data buffer to the second relative destination address in each primary memory unit in the subset of primary memory units corresponding to the broadcast bus, the broadcast bus operating in multicast mode.

7 . The method of claim [ 0012 ]:

wherein, for each processing unit in the set of processing units, transferring the weight tensor partition in the set of weight tensor partitions from the first source address to the first destination address comprises, for each processing unit in the set of processing units:

via a direct memory access core of the processor system, issuing a read request for the weight tensor partition at the first source address;

in response to the read request for the weight tensor partition, loading the weight tensor partition into a data buffer;

selecting a unicast mode for a broadcast bus in the set of broadcast buses corresponding to the primary memory unit of the processing unit; and

transferring the weight tensor partition from the data buffer to the first destination address in the primary memory unit of the processing unit; and

wherein, for each processing unit in the set of processing units, transferring the input tensor partition in the set of input tensor partitions from the fourth source address to a second destination address comprises, for each processing unit in the set of processing units:

via the direct memory access core of the processor system, issuing a read request for the input tensor partition at the fourth source address;

in response to the read request for the input tensor partition, loading the weight tensor partition into the data buffer;

selecting the unicast mode for the broadcast bus in the set of broadcast buses corresponding to the primary memory unit of the processing unit; and

transferring the input tensor partition from the data buffer to the second destination address in the primary memory unit of the processing unit.

8 . The method of claim [ 0012 ]:

wherein generating the output tensor partition of the first output tensor based on the first input tensor and the weight tensor partition comprises executing a first convolution operation based on the first input tensor and the weight tensor partition; and

wherein generating the output tensor partition of the second output tensor based on the second weight tensor and the input tensor partition comprises executing a second convolution operation based on the second weight tensor and the input tensor partition.

9 . A method for executing a neural network by a processor system comprising a shared memory unit, a set of primary memory units, a set of processing units, a direct memory access core, and a broadcast subsystem, the method comprising:

storing, in the shared memory unit:

an input tensor at a first source address, the input tensor characterized by a first set of dimensions; and

a weight tensor at a second source address, the weight tensor:

comprising a set of weight tensor partitions; and

characterized by a second set of dimensions smaller than the first set of dimensions;

via the broadcast subsystem, broadcasting the input tensor from the first source address to a relative destination address in a target set of primary memory units from the set of primary memory units; and

for each primary memory unit in the target set of primary memory units:

transferring a weight tensor partition in the set of weight tensor partitions from the second source address to a destination address in the primary memory unit; and

via a processing unit corresponding to the primary memory unit, generating an output tensor partition of an output tensor based on the input tensor and the weight tensor partition.

10 . The method of claim 9 :

wherein broadcasting the input tensor from the first source address to the relative destination address comprises:

via the direct memory access core, issuing a read request to the shared memory unit for the input tensor at the first source address;

in response to the read request, loading the input tensor into an internal data buffer of the direct memory access core;

via the direct memory access core, issuing a write request comprising:

the relative destination address;

the input tensor from the internal data buffer of the direct memory access core; and

the target set of primary memory units;

via a set of broadcast buses of the broadcast subsystem, transferring the input tensor to the relative destination address in each primary memory unit in the target set of primary memory units.

11 . The method of claim 10 ,

wherein, issuing the write request comprises issuing the write request to a primary memory unit queue in the broadcast subsystem;

wherein transferring the input tensor to the relative destination address in each primary memory unit in the target set of primary memory units comprises:

dequeuing the write request from the primary memory unit queue; and

for each broadcast bus in the set of broadcast buses:

selecting a data transfer mode for the broadcast bus based on the target set of primary memory units; and

via the broadcast bus, transferring the input tensor to the relative destination address according to the data transfer mode.

12 . The method of claim 11 :

wherein selecting the data transfer mode for the broadcast bus based on the target set of primary memory units comprises, in response to the target set of primary memory units comprising a subset of primary memory units corresponding to the broadcast bus, selecting multicast mode for the broadcast bus; and

wherein transferring the input tensor to the relative destination address according to the data transfer mode comprises simultaneously multicasting the input tensor to the subset of primary memory units corresponding to the broadcast bus.

13 . The method of claim 11 :

wherein selecting the data transfer mode for the broadcast bus based on the target set of primary memory units comprises, in response to the broadcast bus partially comprising a subset of primary memory units corresponding to the broadcast bus:

identifying a subset of target primary memory units corresponding to the broadcast bus; and

selecting unicast mode for the broadcast bus; and

wherein transferring the input tensor to the relative destination address according to the data transfer mode comprises serially unicasting the input tensor to the subset of target primary memory units corresponding to the broadcast bus.

14 . The method of claim 9 :

wherein broadcasting the input tensor from the first source address to the first relative destination address comprises:

via the direct memory access core, issuing a read request to the shared memory unit for the input tensor at the first source address;

in response to the read request, loading the input tensor into an internal data buffer of the direct memory access core;

via the direct memory access core, issuing a write request comprising:

the relative destination address;

the input tensor from the internal data buffer of the direct memory access core; and

a selected subset of broadcast buses in the broadcast subsystem and a data transfer mode for each broadcast bus in the selected subset of broadcast buses; and

via the selected subset of broadcast buses of the broadcast subsystem and according to the data transfer mode for each broadcast bus in the selected subset of broadcast buses, transferring the input tensor to the relative destination address in the target set of primary memory units.

15 . A method for executing a neural network by a processor system comprising a shared memory unit, a set of primary memory units, a set of processing units, and a broadcast subsystem, the method comprising:

storing, in the shared memory unit:

a weight tensor at a first source address, the weight tensor characterized by a first set of dimensions; and

an input tensor at a second source address, the input tensor:

comprising a set of input tensor partitions; and

characterized by a second set of dimensions smaller than the first set of dimensions;

broadcasting, via the broadcast subsystem, the weight tensor from the first source address to a relative destination address in a target set of primary memory units in the set of primary memory units; and

for each primary memory unit in the target set of primary memory units:

transferring an input tensor partition in the set of input tensor partitions from the second source address to a destination address in the primary memory unit; and

via a processing unit corresponding to the primary memory unit, generating an output tensor partition of an output tensor based on the weight tensor and the input tensor partition.

16 . The method of claim 15 :

wherein broadcasting the weight tensor from the first source address to the relative destination address comprises:

via the direct memory access core, issuing a read request to the shared memory unit for the weight tensor at the first source address;

in response to the read request, loading the weight tensor into an internal data buffer of the direct memory access core;

via the direct memory access core, issuing a write request comprising:

the relative destination address;

the weight tensor from the internal data buffer of the direct memory access core; and

the target set of primary memory units;

via a set of broadcast buses of the broadcast subsystem, transferring the weight tensor to the relative destination address in each primary memory unit in the target set of primary memory units.

17 . The method of claim 16 :

wherein, issuing the write request comprises issuing the write request to a primary memory unit queue in the broadcast subsystem;

wherein transferring the weight tensor to the relative destination address in each primary memory unit in the target set of primary memory units comprises:

dequeuing the write request from the primary memory unit queue; and

for each broadcast bus in the set of broadcast buses:

selecting a data transfer mode for the broadcast bus based on the target set of primary memory units; and

via the broadcast bus, transferring the weight tensor to the relative destination address according to the data transfer mode.

18 . The method of claim 17 :

wherein selecting the data transfer mode for the broadcast bus based on the target set of primary memory units comprises, in response to the target set of primary memory units comprising a subset of primary memory units corresponding to the broadcast bus, selecting multicast mode for the broadcast bus; and

wherein transferring the weight tensor to the relative destination address according to the data transfer mode comprises simultaneously multicasting the input tensor to the subset of primary memory units corresponding to the broadcast bus.

19 . The method of claim 17 :

wherein selecting the data transfer mode for the broadcast bus based on the target set of primary memory units comprises, in response to the broadcast bus partially comprising a subset of primary memory units corresponding to the broadcast bus:

identifying a subset of target primary memory units corresponding to the broadcast bus; and

selecting unicast mode for the broadcast bus; and

wherein transferring the weight tensor to the relative destination address according to the data transfer mode comprises serially unicasting the input tensor to the subset of target primary memory units corresponding to the broadcast bus.

20 . The method of claim 15 :

wherein broadcasting the input tensor from the first source address to the first relative destination address comprises:

via the direct memory access core, issuing a read request to the shared memory unit for the input tensor at the first source address;

in response to the read request, loading the input tensor into an internal data buffer of the direct memory access core;

via the direct memory access core, issuing a write request comprising:

the relative destination address;

the input tensor from the internal data buffer of the direct memory access core; and

a selected subset of broadcast buses in the broadcast subsystem and a data transfer mode for each broadcast bus in the selected subset of broadcast buses;

via the selected subset of broadcast buses of the broadcast subsystem and according to the data transfer mode for each broadcast bus in the selected subset of broadcast buses, transferring the input tensor to the relative destination address in the target set of primary memory units.