Dynamic routing for accelerated deep learning
Techniques in dynamic routing for advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements comprising a portion of a neural network accelerator performs flow-based computations on wavelets of data. Each processing element comprises a compute element enabled to execute programmed instructions using the data and a router enabled to route the wavelets via static routing, dynamic routing, or both. The routing is in accordance with a respective virtual channel specifier of each of the wavelets and controlled by routing configuration information of the router. The static techniques enable statically specifiable neuron connections. The dynamic techniques enable information from the wavelets to alter the routing configuration information during neural network processing.
1. A method comprising:
exchanging packets between a plurality of processing elements coupled via a fabric, each processing element comprising a respective router, wherein each router is coupled to the fabric via a respective plurality of fabric ports, a particular one of the processing elements comprises a particular one of the routers, the particular router is coupled to the fabric via a particular one of the pluralities of fabric ports, and one or more of the exchanged packets comprises a respective index field;
transmitting, as one of the one or more exchanged packets, a particular packet onto the fabric via a first fabric port of the particular plurality of fabric ports as indicated by a first mask from a switch that is configurable to select the first mask from a plurality of masks, wherein the index field of the transmitted packet is a modification of an index value;
modifying at least a first sub-field of the index value based on a second sub-field of the index value to formulate the modification, wherein the first sub-field and the second sub-field are distinct sub-fields of the index value; and
configuring the switch, responsive to the first sub-field indicating a first one of a plurality of switch opcodes, to select a second one of the plurality of masks that is different from the first mask such that a subsequent packet from the switch is transmitted onto the fabric via a second fabric port of the particular plurality of fabric ports as indicated by the second mask.
2. The method of claim 1 , wherein the particular packet is a first packet; and further comprising communicating, as one of the exchanged packets, a second packet with the fabric via the first fabric port.
3. The method of claim 1 , wherein the modifying is conditional on which of the plurality of masks the switch is configured to select.
4. The method of claim 1 , wherein the modifying is conditional on operating in a first one of a plurality of mutually exclusive modes.
5. The method of claim 1 , wherein the configuring is a first configuring and further comprising a second configuring of the switch, responsive to the first sub-field indicating a second one of the plurality of switch opcodes, to select an initial one of the plurality of masks.
6. The method of claim 1 , wherein:
the particular packet is a first packet,
each exchanged packet further comprises a respective color field, the color field of the particular packet is a first color value, and the color field of a second packet is a second color value,
the switch is a first switch that corresponds to the first color value, and
the plurality of masks is a first plurality of masks that corresponds to the first color value; and
further comprising communicating, as one of the exchanged packets, the second packet with the fabric via a second fabric port of the particular plurality of fabric ports as indicated by a second mask from a second switch that corresponds to the second color value and that is configurable to select the second mask from a second plurality of masks.
7. The method of claim 1 , wherein each exchanged packet further comprises a respective color field and the color field of the particular packet is a color value.
8. The method of claim 7 , further comprising selecting one of a plurality of queues, the queue selecting based at least in part on the color value, and storing at least the index value into the selected queue.
9. The method of claim 8 , wherein each processing element further comprises a respective compute element and the particular one of the processing elements further comprises a particular one of the compute elements; and further comprising executing one or more programmed instructions via the particular compute element to configure the switch to provide the first mask.
10. The method of claim 8 , wherein each processing element further comprises a respective compute element, the particular one of the processing elements further comprises a particular one of the compute elements, and the configuring is conditional on the switch operating in a first one of a plurality of mutually exclusive operating modes; and further comprising executing one or more programmed instructions via the particular compute element to initialize the switch to operate in the first operating mode.
11. The method of claim 10 , wherein the configuring is further conditional on a value of an indicator field of the particular packet that is distinct from the first sub-field.
12. A method comprising:
exchanging packets between a plurality of processing elements coupled via a fabric, each processing element comprising a respective router, wherein each router is coupled to the fabric via a respective plurality of fabric ports, a particular one of the processing elements comprises a particular one of the routers, the particular router is coupled to the fabric via a particular one of the pluralities of fabric ports, and one or more of the exchanged packets comprises a respective index field;
transmitting, as one of the one or more exchanged packets, a particular packet onto the fabric via a group of fabric ports as identified by a determined mask of a plurality of masks, wherein the determined mask is based at least in part on a control value, each of the plurality of masks identifies a respective one or more of the fabric ports of the particular plurality of fabric ports as the group of fabric ports, and the index field of the transmitted packet is a modification of an index value;
modifying at least a first sub-field of the index value based on a second sub-field of the index value to formulate the modification, wherein the first sub-field and the second sub-field are distinct sub-fields of the index value; and
updating the control value, responsive to the first sub-field indicating a first one of a plurality of opcodes, wherein the updated control value corresponds to another mask of the plurality of masks, the another mask indicating another group of fabric ports for transmitting a subsequent packet onto the fabric.
13. The method of claim 12 , wherein the particular packet is a first packet; and further comprising communicating, as one of the exchanged packets, a second packet with the fabric via the group of fabric ports.
14. The method of claim 12 , wherein:
the particular packet is a first packet,
each exchanged packet further comprises a respective color field, the color field of the particular packet is a first color value, and the color field of a second packet is a second color value,
the group of fabric ports is a first group of fabric ports, and
the plurality of masks is a first plurality of masks that corresponds to the first color value; and
further comprising communicating, as one of the exchanged packets, the second packet with the fabric via a second group of fabric ports as identified by a second mask from a second plurality of masks that corresponds to the second color value.
15. The method of claim 1 or claim 12 , wherein a portion of the exchanged packets comprise respective data fields and one or more of the data fields indicate one or more of an activation of a neural network, a partial sum of activations of a neural network, an error of a neural network, a gradient estimate of a neural network, and a weight of a neural network.
16. The method of claim 1 or claim 12 , wherein the processing elements and the fabric are implemented on a substantially whole wafer.
17. The method of claim 2 or claim 13 , wherein the communicating comprises one of transmitting and receiving.
18. The method of claim 6 or claim 14 , wherein the first color value and the second color value specify respective communication pathways.
19. The method of claim 1 or claim 12 , wherein each processing element further comprises a respective compute element and the particular one of the processing elements further comprises a particular one of the compute elements.
20. The method of claim 19 , further comprising executing a programmed instruction via the particular compute element to produce the index value.
21. A method of communicating packets between a plurality of processing elements using a fabric, each processing element of the plurality of processing elements comprising a respective router coupled to the fabric using a respective plurality of fabric ports, the method comprising:
transmitting, using a first fabric port identified by an output switch of a router, a packet onto the fabric, wherein the packet comprises an index field, the index field comprising a first sub-field and a second sub-field, the first sub-field indicative of a switch opcode;
modifying at least the first sub-field based on the second sub-field; and
configuring the output switch of the router to select a second fabric port different from the first fabric port based on the switch opcode.