Neural network processing element of accelerator tile
Described is a neural network accelerator tile. It includes an activation memory interface for interfacing with an activation memory to receive a set of activation representations and a weight memory interface for interfacing with a weight memory to receive a set of weight representations, and a processing element. The processing element is configured to implement a one-hot encoder, a histogrammer, an aligner, a reducer, and an accumulation sub-element which process the set of activation representations and the set of weight representations to produce a set of output representations.
1. A neural network accelerator tile, comprising:
an activation memory interface for interfacing with an activation memory to receive a set of activation representations, wherein the set of activation representations is represented by an exponential expression based on activation exponent values;
a weight memory interface for interfacing with a weight memory to receive a set of weight representations, wherein the set of weight representations is represented by an exponential expression based on weight exponent values; and
a processing element configured to implement an exponent sub-element, a one-hot encoder, a histogrammer, an aligner, a reducer, and an accumulation sub-element to process the set of activation representations and the set of weight representations to produce a set of output representations,
wherein the exponent sub-element performs addition operations using the activation exponent values and the weight exponent values to generate exponent values.
2. The accelerator tile of claim 1 , wherein the activation memory interface is configured to provide the set of activation representations to the processing element as a set of activation one-offset pairs, and the weight memory interface is configured to provide the set of weight representations to the processing element as a set of weight one-offset pairs.
3. The accelerator tile of claim 2 , wherein the exponent sub-element is configured to combine the set of activation one-offset pairs and the set of weight one-offset pairs to produce a set of one-offset pair products.
4. The accelerator tile of claim 3 , wherein the exponent sub-element includes a set of magnitude adders and a corresponding set of sign gates, one pair of magnitude adder and sign gate to provide each one-offset pair product of the set of one-offset pair products.
5. The accelerator tile of claim 1 , wherein the one-hot encoder includes a set of decoders to perform one-hot encoding of a set of one-hot encoder inputs.
6. The accelerator tile of claim 2 , wherein the histogrammer is configured to sort a set of histogrammer inputs by value.
7. The accelerator tile of claim 2 , wherein the aligner is configured to shift a set of aligner inputs to provide a set of shifted inputs to be reduced.
8. The accelerator tile of claim 2 , wherein the reducer includes an adder tree to reduce a set of reducer inputs into a partial sum.
9. The accelerator tile of claim 2 , wherein the accumulation sub-element includes an accumulator and is configured to receive a partial sum and add the partial sum to the accumulator to accumulate a product over multiple cycles.
10. The accelerator tile of claim 1 , wherein the processing element is further configured to implement a concatenator.
11. The accelerator tile of claim 10 , wherein the concatenator is provided to concatenate a set of concatenator inputs to produce a set of grouped counts to be shifted and reduced by the aligner and reducer to produce a partial product.
12. A neural network comprising a set of neural network accelerator tiles according to claim 1 .
13. The neural network of claim 12 , wherein the neural network is implemented as a convolutional neural network.
14. The neural network of claim 12 , wherein the set of neural network accelerator tiles includes a column of accelerator tiles, the tiles of the column of accelerator tiles configured to each receive the same set of activation representations and to each receive a unique set of weight representations.
15. The accelerator tile of claim 1 , wherein the processing element is configured to, over multiple cycles, process an activation representation of the set of activation representations with a weight representation of the set of weight representations.
16. A method of producing a neural network partial product, implementing processing components including an exponent sub-element, a one-hot encoder, a histogrammer, an aligner and a reducer, the method comprising:
receiving a set of activation representations, wherein the set of activation representations is represented by an exponential expression based on activation exponent values;
receiving a set of weight representations, each weight representation corresponding to an activation representation of the set of activation representations, wherein the set of weight representations is represented by an exponential expression based on weight exponent values;
generating, by the exponent sub-element, a set of partial exponent value results by combining the set of weight representations with the set of activation representations by combining each weight representation with its corresponding activation representation;
generating, by the one-hot encoder, a set of one-hot representations by encoding the set of partial exponent value results;
accumulating, by the histogrammer, the set of one-hot representations into a set of histogram bucket counts;
aligning, by the aligner, the counts of the set of histogram bucket counts according to their size; and
generating, by the reducer, the neural network partial product by reducing the aligned counts of the set of histogram bucket counts,
wherein the combination of the set of weight representations with the set of activation representations is based on an addition operation using the activation exponent values and the weight exponent values, and
wherein the aligner and the reducer operate based on an output, of the histogrammer, that is generated by the histogrammer to not ha q non-zero bits.
17. The method of claim 16 , further comprising outputting the neural network partial product to an accumulator to accumulate a product.
18. The method of claim 17 , further comprising outputting the product to an activation memory.
19. The method of claim 16 , further comprising, prior to aligning the counts of the set of histogram bucket counts, recursively grouping the counts of the set of histogram bucket counts into a set of grouped counts and providing the set of grouped counts to be aligned and reduced in place of the set of histogram bucket counts.
20. The method of claim 16 , wherein combining a weight representation of the set of weight representations with an activation representation of the set of activation representations is performed over multiple cycles.