Dynamic routing for accelerated deep learning
Techniques in dynamic routing for advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements comprising a portion of a neural network accelerator performs flow-based computations on wavelets of data. Each processing element comprises a compute element enabled to execute programmed instructions using the data and a router enabled to route the wavelets via static routing, dynamic routing, or both. The routing is in accordance with a respective virtual channel specifier of each of the wavelets and controlled by routing configuration information of the router. The static techniques enable statically specifiable neuron connections. The dynamic techniques enable information from the wavelets to alter the routing configuration information during neural network processing.
1 . A method comprising:
transmitting, using a first fabric port of a first processing element, a wavelet to a second fabric port of a second processing element using a fabric coupled to the first fabric port and to the second fabric port, wherein the wavelet comprises information indicating a first subsequent fabric port of the first processing element;
determining, using the first processing element, a second subsequent fabric port of a subsequent processing element to send a subsequent wavelet using the first subsequent fabric port; and
transmitting, using the first subsequent fabric port, the subsequent wavelet to the second subsequent fabric port of he subsequent processing element using the fabric, the fabric coupled to the first subsequent fabric port and to the second subsequent fabric port.
2 . The method of claim 1 , further comprising processing the wavelet by a first compute element of the first processing element, wherein the information further indicates that the wavelet has been processed by the first processing element.
3 . The method of claim 1 , wherein the first fabric port and the first subsequent fabric port are different.
4 . The method of claim 3 , wherein the information further indicates switching from a first routing pattern to a second routing pattern associated with the first subsequent fabric port and the second subsequent fabric port, respectively.
5 . The method of claim 4 , wherein the information further indicates that the first routing pattern is associated with a first color value, and that the second routing pattern is associated with a second color value.
6 . The method of claim 1 , wherein the first processing element comprises a router, wherein the first fabric port is identified by an output switch of the router, and wherein transmitting, using the first subsequent fabric port, the subsequent wavelet to the second subsequent fabric port comprises configuring the output switch of the router to select the first subsequent fabric port.
7 . The method of claim 1 , wherein the wavelet comprises a first field and a second field, and wherein the first field is for storing a first opcode, the method further comprising modifying, based at least in part on the second field, the first field to determine a second opcode associated with the first subsequent fabric port.
8 . The method of claim 7 , wherein modifying, based at least in part on the second field, the first field to determine the second opcode associated with the first subsequent fabric port comprises shifting one or more bits of the wavelet to the first field.
9 . The method of claim 1 , wherein the subsequent wavelet is a first subsequent wavelet, wherein the information further indicates a third fabric port of the first processing element, the method further comprising selecting the third fabric port for receiving a second subsequent wavelet.
10 . The method of claim 1 , wherein the information further indicates one or more of an activation of a neural network, a partial sum of activations of a neural network, an error of a neural network, a gradient estimate of a neural network, or a weight of a neural network.
11 . A system comprising:
a plurality of processing elements, each processing element comprising one or more respective fabric ports; and
a fabric coupled to the one or more respective fabric ports of the plurality of processing elements, wherein a first processing element of the plurality of processing elements is configured to:
transmit, using a first fabric port of the first processing element, a wavelet to a second fabric port of a second processing element of the plurality of processing elements using the fabric, wherein the wavelet comprises information indicating a first subsequent fabric port of the first processing element;
determine a second subsequent fabric port of a subsequent processing element to send a subsequent wavelet using the first subsequent fabric port; and
transmit, using the first subsequent fabric port, the subsequent wavelet to the second subsequent fabric port of the subsequent processing element of the plurality of processing elements using the fabric.
12 . The system of claim 11 , wherein the first processing element is further configured to process the wavelet by a first compute element of the first processing element, wherein the information further indicates that the wavelet has been processed by the first processing element.
13 . The system of claim 11 , wherein the first fabric port and the first subsequent fabric port are different.
14 . The system of claim 13 , wherein the information further indicates switching from a first routing pattern to a second routing pattern associated with the first subsequent fabric port and the second subsequent fabric port, respectively.
15 . The system of claim 14 , wherein the information further indicates that the first routing pattern is associated with a first color value and that the second routing pattern is associated with a second color value.
16 . The system of claim 11 , wherein the first processing element comprises a router, wherein the first fabric port is identified by an output switch of the router, and wherein the first processing element is configured to transmit, using the first subsequent fabric port, the subsequent wavelet to the second subsequent fabric port by configuring the output switch of the router to select the first subsequent fabric port.
17 . The system of claim 11 , wherein the wavelet comprises a first field and a second field, and wherein the first field is for storing a first opcode, and wherein the first processing element is further configured to modify, based at least in part on the second field, the first field to determine a second opcode associated with the first subsequent fabric port.
18 . The system of claim 17 , wherein the first processing element is configured to modify, based at least in part on the second field, the first field to determine the second opcode associated with the first subsequent fabric port by shifting one or more bits of the wavelet to the first field.
19 . The system of claim 11 , wherein the subsequent wavelet is a first subsequent wavelet, wherein the information further indicates a third fabric port of the first processing element, and wherein the first processing element is further configured to select the third fabric port for receiving a second subsequent wavelet.
20 . The system of claim 11 , wherein the information further indicates one or more of an activation of a neural network, a partial sum of activations of a neural network, an error of a neural network, a gradient estimate of a neural network, or a weight of a neural network.