Data structure descriptors for deep learning acceleration
Techniques in advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements performs flow-based computations on wavelets of data. Each processing element has a respective compute element and a respective routing element. Instructions executed by the compute element include operand specifiers, some specifying a data structure register storing a data structure descriptor describing an operand as a fabric vector or a memory vector. The data structure descriptor further describes the memory vector as one of a one-dimensional vector, a four-dimensional vector, or a circular buffer vector. Optionally, the data structure descriptor specifies an extended data structure register storing an extended data structure descriptor. The extended data structure descriptor specifies parameters relating to a four-dimensional vector or a circular buffer vector.
1. A method comprising:
in a compute element, decoding an instruction, the instruction comprising an operand field;
in the compute element, accessing an operand descriptor based at least in part on the operand field;
in the compute element, decoding the operand descriptor to determine a particular one of a plurality of mutually exclusive types the operand descriptor refers to;
in the compute element, accessing an operand in accordance with the operand descriptor and the particular type;
performing an iteration of the instruction via accessing, in accordance with an access pattern described by the operand descriptor, sufficient data elements of a vector for the iteration;
wherein the mutually exclusive types comprise a fabric type and a memory type;
wherein the compute element is comprised in a processing element that comprises a fabric router, and the processing element is one of a plurality of processing elements each comprising a respective compute element and a respective fabric router;
wherein the processing elements are interconnected via a fabric coupled to the respective fabric routers;
wherein responsive to the particular type being the fabric type, the operand is accessed via the fabric; and
wherein responsive to the particular type being the memory type, the operand is accessed via a memory of the compute element.
2. The method of claim 1 , further comprising performing an iteration of the instruction via accessing, in accordance with an access pattern described by the operand descriptor, sufficient data elements of a vector for the iteration.
3. The method of claim 2 , wherein the access pattern is one of a fabric vector, a one-dimensional memory vector, a four-dimensional memory vector, and a circular memory buffer.
4. The method of claim 1 , wherein the operand descriptor comprises microthreading information describing how the compute element is to operate responsive to the operand being a vector and there is a stall accessing an element of the vector.
5. The method of claim 4 , further comprising, responsive to the stall and the microthreading information indicating microthreading is not enabled, the computer element stalling.
6. The method of claim 4 , wherein the instruction is a first instruction, and further comprising, responsive to the stall and the microthreading information indicating microthreading is enabled, the compute element suspends processing of the first instruction and selects a second instruction for processing.
7. The method of claim 6 , wherein the first instruction is associated with a first task and the second instruction is associated with a second task.
8. The method of claim 1 , wherein the operand is a vector and the operand descriptor indicates how many elements of the vector to process in parallel.
9. The method of claim 1 , wherein the operand descriptor is enabled to specify a vector length of vector operands.
10. The method of claim 1 , wherein the operand descriptor comprises an indicator of whether to terminate processing responsive to the operand being a vector and a control fabric packet conveying an element of the vector being received via the fabric.
11. The method of claim 1 , wherein the operand descriptor comprises an indicator of a virtual channel to selectively activate responsive to completion of the instruction.
12. The method of claim 1 , wherein responsive to the type being the fabric type and the operand being a source, the accessing the operand comprises reading data elements from an input queue coupled to the fabric.
13. The method of claim 1 , wherein responsive to the type being the fabric type and the operand being a destination, the accessing the operand comprises writing data elements to an output queue coupled to the fabric.
14. The method of claim 1 , wherein execution of the instruction implements at least a portion of any one or more of: computing an activation of a neural network, computing a partial sum of activations of a neural network, computing an error of a neural network, computing a gradient estimate of a neural network, and updating a weight of a neural network.
15. The method of claim 1 , wherein the operand comprises at least a portion of any one or more of: a weight of a neural network, an activation of a neural network, a partial sum of activations of a neural network, an error of a neural network, a gradient estimate of a neural network, and a weight update of a neural network.
16. The method of claim 1 , wherein a substantially whole wafer comprises the processing elements and the fabric.
17. A system comprising:
a plurality of processing elements, each processing element comprising a fabric router coupled to a compute element;
wherein each processing element is enabled to selectively communicate fabric packets with others of the processing elements at least in part via the fabric router of the respective processing element; and
wherein each compute element comprises a memory and is enabled to
decode an instruction comprising an operand field,
access an operand descriptor based at least in part on the operand field,
decode the operand descriptor to determine a particular one of a plurality of mutually exclusive types the operand descriptor refers to, the mutually exclusive types comprising a fabric type and a memory type,
access an operand in accordance with the operand descriptor and the particular type,
wherein the access of the operand is via the respective fabric router coupled to the compute element responsive to the particular type being the fabric type, and
wherein the access of the operand is via the memory responsive to the particulate type being the memory type.
18. The system of claim 17 , wherein each compute element is further enabled, responsive to the type being the fabric type and the operand being a source, to access the operand via reading data elements from an input queue coupled to the fabric.
19. The system of claim 17 , wherein each compute element is further enabled, responsive to the type being the fabric type and the operand being a destination, to access the operand via writing data elements to an output queue coupled to the fabric.
20. The system of claim 17 , wherein the operand descriptor is enabled to identify one of a plurality of extended operand descriptors.
21. The system of claim 20 , wherein the extended operand descriptors are enabled to specify one or more of stride information and dimension information of a four-dimensional memory vector.
22. The system of claim 20 , wherein the extended operand descriptors are enabled to specify one or more of a start address and an end address of a circular memory buffer.
23. The system of claim 20 , wherein the extended operand descriptors are enabled to specify FIFO or non-FIFO operation of a circular memory buffer.
24. The system of claim 17 , wherein execution of the instruction implements at least a portion of any one or more of: computing an activation of a neural network, computing a partial sum of activations of a neural network, computing an error of a neural network, computing a gradient estimate of a neural network, and updating a weight of a neural network.
25. The system of claim 17 , wherein the operand comprises at least a portion of any one or more of: a weight of a neural network, an activation of a neural network, a partial sum of activations of a neural network, an error of a neural network, a gradient estimate of a neural network, and a weight update of a neural network.
26. The system of claim 17 , wherein a substantially whole wafer comprises the processing elements.
27. A system comprising:
means for decoding an instruction, the instruction comprising an operand field;
means for accessing an operand descriptor based at least in part on the operand field;
means for decoding the operand descriptor to determine a particular one of a plurality of mutually exclusive types the operand descriptor refers to;
means for accessing an operand in accordance with the operand descriptor and the particular type;
means for performing an iteration of the instruction via accessing, in accordance with an access pattern described by the operand descriptor, sufficient data elements of a vector for the iteration;
wherein the mutually exclusive types comprise a fabric type and a memory type;
wherein a compute element comprises the means for decoding an instruction, the means for accessing an operand descriptor, the means for decoding the operand descriptor, the means for accessing an operand, and the means for performing an iteration;
wherein the compute element is comprised in a processing element that comprises a fabric router, and the processing element is one of a plurality of processing elements each comprising a respective compute element and a respective fabric router;
wherein the processing elements are interconnected via a fabric coupled to the respective fabric routers;
wherein responsive to the particular type being the fabric type, the operand is accessed via the fabric; and
wherein responsive to the particular type being the memory type, the operand is accessed via a memory of the compute element.
28. The system of claim 27 , wherein execution of the instruction implements at least a portion of any one or more of: computing an activation of a neural network, computing a partial sum of activations of a neural network, computing an error of a neural network, computing a gradient estimate of a neural network, and updating a weight of a neural network.
29. The system of claim 27 , wherein the operand comprises at least a portion of any one or more of: a weight of a neural network, an activation of a neural network, a partial sum of activations of a neural network, an error of a neural network, a gradient estimate of a neural network, and a weight update of a neural network.
30. The system of claim 27 , wherein a substantially whole wafer comprises the processing elements and the fabric.