Efficient neural network accelerator dataflows
A distributed deep neural net (DNN) utilizing a distributed, tile-based architecture includes multiple chips, each with a central processing element, a global memory buffer, and a plurality of additional processing elements. Each additional processing element includes a weight buffer, an activation buffer, and vector multiply-accumulate units to combine, in parallel, the weight values and the activation values using stationary data flows.
1. A data processor comprising:
a plurality of processing elements, each comprising:
a weight buffer;
an activation buffer;
an accumulation memory buffer;
a plurality of vector multiply-accumulate units configured to compute in parallel a convolution of weights from the weight buffer and activations from the activation buffer, each of the vector multiply-accumulate units comprising:
a first collector disposed between the vector multiply-accumulate unit and the weight buffer; and
a second collector disposed between the vector multiply-accumulate unit and the accumulation memory buffer; and
configuration logic to configure a depth of the collectors to adjust a level of data-stationary computation of the convolution.
2. The data processor of claim 1 , further comprising:
the configuration logic to configure a depth of the first collector on one or more of the vector multiply-accumulate units to adjust a level of weight-stationary computation of the convolution.
3. The data processor of claim 1 , further comprising:
the configuration logic to configure a depth of the second collector on one or more of the vector multiply-accumulate units to adjust a level of output-stationary computation of the convolution.
4. The data processor of claim 1 , further comprising:
the configuration logic to configure one or both of a depth of the first collector and a depth of the second collector on one or more of the vector multiply-accumulate units to implement one or more of multi-level weight-stationary computation and output-stationary computation of the convolution.
5. The data processor of claim 1 , the processing elements further comprising:
a third collector disposed between the activation buffer and the vector multiply-accumulate units.
6. The data processor of claim 5 , further comprising:
the configuration logic to configure a depth of the third collector to adjust a level of input-stationary computation of the convolution.
7. The data processor of claim 5 , further comprising:
the configuration logic to configure one or more of a depth of the first collector, a depth of the second collector, and a depth of the third collector of one or more of the vector multiply-accumulate units to implement one or more of multi-level weight-stationary, output-stationary, and input stationary computation of the convolution.
8. The data processor of claim 1 , further comprising:
a global memory; buffer; and
the configuration logic to configure the processing elements to utilize the global memory buffer to store the activations and to apply the activations between layers of a neural network.
9. The data processor of claim 1 , further comprising:
the configuration logic to configure one or more of the vector multiply-accumulate units to compute a portion of the convolution as a partial result and to forward the partial result from the accumulation memory buffer to neighboring processing elements.
10. The data processor of claim 1 , further comprising:
the configuration logic to distribute the weights and the activations among the processing elements spatially by a depth of an input of a neural network, and temporally by a height and a width of the input to the neural network.
11. The data processor of claim 1 , further comprising:
the configuration logic to distribute the weights and the activations among the processing elements spatially and temporally by configurable combinations of input dimensions of a neural network and dimensions of the weights.
12. A neural network computation method comprising:
distributing weight values and activation values for a neural network computation among a plurality of processing elements of the neural network spatially by a depth of an input to the neural network, and temporally by a height and a width of the input to the neural network;
configuring a depth of at least one collector of a plurality of vector multiply-accumulate units of the processing elements to implement a stationary data flow by the vector multiply-accumulate units during the neural network computation.
13. The neural network computation method of claim 12 , wherein the stationary data flow is a weigh-stationary data flow.
14. The neural network computation method of claim 12 , wherein the data flow is an output-stationary data flow.
15. A neural network computation method comprising:
distributing weight values and activation values for a neural network computation among a plurality of processing elements of the neural network spatially and temporally by configurable combinations of different dimensions of inputs to the neural network and weights of the neural network; and
configuring a depth of at least one collector of a plurality of vector multiply-accumulate units of a plurality of processing elements to implement a stationary data flow by the vector multiply-accumulate units during the neural network computation.