Networks for distributing parameters and data to neural network compute cores
Networks for distributing parameters and data to neural network compute cores. In various embodiments, a neural inference chip comprises a plurality of neural cores and at least one network interconnecting the plurality of neural cores. Each of the plurality of neural cores is adapted to apply a plurality of synaptic weights to a plurality of input activations to produce a plurality of output activations. The at least one network is adapted to simultaneously deliver synaptic weights and/or input activations to the plurality of neural cores.
1. A neural inference chip comprising:
a plurality of neural cores,
each of the plurality of neural cores adapted to apply a plurality of synaptic weights to a plurality of input activations to produce a plurality of output activations;
at least one network interconnecting the plurality of neural cores,
the at least one network adapted to simultaneously deliver synaptic weights and activation function parameters, and simultaneously deliver input activations to the plurality of neural cores based on an arrangement of the plurality of neural cores; and
a plurality of queues, each queue of the plurality of queues associated with one of the plurality of neural cores,
the plurality of queues adapted to receive the synaptic weights and activation function parameters and facilitate the simultaneous delivery of the synaptic weights and activation function parameters.
2. The neural inference chip of claim 1 , wherein the plurality of cores is organized in a grid of two or more dimensions with at least one row and at least one column.
3. The neural inference chip of claim 2 , wherein the at least one network comprises a row or column bus.
4. The neural inference chip of claim 2 , wherein the at least one network comprises a row or column tree.
5. The neural inference chip of claim 2 , wherein the at least one network comprises a systolic row or column shifter.
6. The neural inference chip of claim 5 , wherein the systolic row or column shifter is adapted to deliver a data tensor or block sequentially to cores coupled to the systolic row or column shifter.
7. The neural inference chip of claim 6 , wherein the data tensor or block comprises neural network input activations, intermediate activations, and/or parameters.
8. The neural inference chip of claim 5 , wherein the at least one network comprises at least two systolic row or column shifters configured to send data from opposite directions in the at least one network.
9. The neural inference chip of claim 2 , wherein the grid comprises a mesh.
10. The neural inference chip of claim 9 , wherein the mesh is adapted to communicate data between cores in cardinal directions.
11. The neural inference chip of claim 1 , wherein the at least one network comprises a plurality of branches arranged in a tree network topology, and buffers disposed at each of the plurality of branches.
12. The neural inference chip of claim 1 , wherein the at least one network comprises a plurality of branches arranged in a tree network topology, and routing controls disposed at each of the plurality of branches.
13. The neural inference chip of claim 1 , wherein the at least one network is adapted to deliver intermediate results among the plurality of cores.
14. The neural inference chip of claim 13 , wherein the intermediate results comprise partial sums.
15. The neural inference chip of claim 14 , wherein the partial sums comprise weighted sums of a subset of inputs.
16. The neural inference chip of claim 1 , wherein the at least one network comprises a broadcast network.
17. The neural inference chip of claim 16 , wherein the broadcast network is adapted to deliver a data tensor or block to all cores coupled to the broadcast network.
18. The neural inference chip of claim 17 , wherein the data tensor or block comprises neural network input activations, intermediate activations, and/or parameters.
19. The neural inference chip of claim 1 , wherein the at least one network comprises a multicast network.
20. The neural inference chip of claim 19 , wherein the multicast network is adapted to deliver a data tensor or block to a subset of cores coupled to the broadcast network.
21. The neural inference chip of claim 20 , wherein the data tensor or block comprises neural network input activations, intermediate activations, and/or parameters.
22. A method comprising:
by at least one network of a neural inference chip,
simultaneously delivering synaptic weights and activation function parameters, and simultaneously delivering input activations to a plurality of neural cores of the neural inference chip;
by each of the plurality of neural cores,
applying a plurality of synaptic weights to a plurality of input activations to produce a plurality of output activations, and sending the output activations via the at least one network; and
by a plurality queues, each of the plurality of queues being associated with one of the plurality of neural cores,
receiving the synaptic weights and activation function parameters, and
facilitating the simultaneous delivery of the synaptic weights and activation function parameters.