Sparse convolutional neural network accelerator
A method, computer program product, and system perform computations using a sparse convolutional neural network accelerator. Compressed-sparse data is received for input to a processing element, wherein the compressed-sparse data encodes non-zero elements and corresponding multi-dimensional positions. The non-zero elements are processed in parallel by the processing element to produce a plurality of result values. The corresponding multi-dimensional positions are processed in parallel by the processing element to produce destination addresses for each result value in the plurality of result values. Each result value is transmitted to a destination accumulator associated with the destination address for the result value.
1. A method, comprising:
receiving compressed-sparse data for input to at least one processing element included in a plurality of processing elements, wherein the compressed-sparse data encodes non-zero elements and corresponding multi-dimensional positions;
processing, via a multiplier array within the at least one processing element, the non-zero elements separately from the corresponding multi-dimensional positions to produce a plurality of result values;
decoding the corresponding multi-dimensional positions separately from the non-zero elements to produce a destination multi-dimensional position for each result value in the plurality of result values, wherein the destination multi-dimensional position is associated with a respective destination accumulator of a plurality of destination accumulators within an accumulator array and each processing element of the plurality of processing elements is coupled to each one of the plurality of destination accumulators through a scatter network; and
transmitting each result value through the scatter network to the respective destination accumulator associated with the destination multi-dimensional position for the result value.
2. The method of claim 1 , wherein the non-zero elements are weight values.
3. The method of claim 1 , wherein the non-zero elements are input activation values.
4. The method of claim 1 , wherein the multi-dimensional positions are encoded as a sequence of zero-counts and each zero-count is a number of zeros between pairs of the non-zero elements.
5. The method of claim 4 , wherein the decoding of the corresponding multi-dimensional positions comprises:
converting the zero-counts into coordinates of the multi-dimensional positions; and
for each result value, summing the coordinates in each dimension of the multi-dimensional positions to produce coordinates of the destination multi-dimensional position associated with one of the plurality of destination accumulators.
6. The method of claim 1 , wherein the multi-dimensional positions are encoded as a sequence of codes and each code defines a zero-count that is a number of zeros between pairs of the non-zero elements.
7. The method of claim 6 , wherein a first code of the codes defines a number of rows to skip in a first coordinate dimension associated with the multi-dimensional positions.
8. The method of claim 6 , wherein the codes define a linear progression of the zero-counts.
9. The method of claim 6 , wherein the codes define a non-linear progression of the zero-counts.
10. The method of claim 1 , wherein the decoding of the corresponding multi-dimensional positions comprises, for each result value, summing coordinates in each dimension of the multi-dimensional position to produce coordinates of the destination multi-dimensional position associated with one of the plurality of destination accumulators.
11. The method of claim 1 , wherein the destination multi-dimensional positions are three-dimensional positions.
12. A convolutional neural network accelerator, comprising:
an array of processing elements, wherein at least one processing element in the array of processing elements receives compressed-sparse data that encodes non-zero elements and corresponding multi-dimensional positions, the at least one processing element comprising:
a plurality of destination accumulators within an accumulator array;
an array of multipliers configured to process the non-zero elements separately from the corresponding multi-dimensional positions to produce a plurality of result values, wherein the corresponding multi-dimensional positions are decoded separately from the non-zero elements to produce a destination multi-dimensional position for each result value in the plurality of result values, wherein the destination multi-dimensional position is associated with a respective destination accumulator of the plurality of destination accumulators; and
a scatter network coupling each multiplier in the array of multipliers to each one of the plurality of destination accumulators and configured to transmit each result value to the respective destination accumulator associated with the destination multi-dimensional position for the result value.
13. The convolutional neural network accelerator of claim 12 , wherein the multi-dimensional positions are encoded as a sequence of zero-counts and each zero-count is a number of zeros between pairs of the non-zero elements.
14. The convolutional neural network accelerator of claim 12 , wherein the decoding of the corresponding multi-dimensional positions comprises,
for each result value, summing coordinates in each dimension of the multi-dimensional position to produce coordinates of the destination multi-dimensional position associated with one of the plurality of destination accumulators.
15. The convolutional neural network accelerator of claim 12 , wherein the multi-dimensional positions are encoded as a sequence of codes and each code defines a zero-count that is a number of zeros between pairs of the non-zero elements.
16. A system, comprising:
a memory storing compressed-sparse data that encodes non-zero elements and corresponding multi-dimensional positions; and
an array of processing elements, wherein at least one processing element in the array of processing elements comprises:
a plurality of destination accumulators within an accumulator array;
an array of multipliers configured to process the non-zero elements without the corresponding multi-dimensional positions to produce a plurality of result values, wherein the corresponding multi-dimensional positions are decoded separately from the non-zero elements to produce a destination multi-dimensional position for each result value in the plurality of result values, wherein the destination multi-dimensional position is associated with a respective destination accumulator of the plurality of destination accumulators; and
a scatter network coupling each multiplier in the array of multipliers to each one of the plurality of destination accumulators and configured to transmit each result value to the respective destination accumulator associated with the destination multi-dimensional position for the result value.
17. The method of claim 1 , further comprising:
computing accumulated values by the destination accumulator and additional destination accumulators in the accumulator array that are associated with other destination multi-dimensional positions by summing the result values received at each destination accumulator;
removing accumulated values of zero from the accumulated values; and
compressing the accumulated values to produce compressed-sparse activations.
18. The method of claim 1 , wherein the non-zero elements and the corresponding multi-dimensional positions are processed in parallel.
19. The method of claim 1 , wherein a first number of result values in the plurality of result values is not equal to a second number of the destination accumulators in the accumulator array.
20. The system of claim 16 , wherein the decoding of the corresponding multi-dimensional positions comprises, for each result value, summing coordinates in each dimension of the multi-dimensional positions to produce coordinates of the destination multi-dimensional position associated with one of the plurality of destination accumulators.