Vector operation acceleration with convolution computation unit

1. A vector operation accelerator for neural network computations,

comprises:

a plurality of weight selectors configured to obtain weights;

a plurality of activation input interfaces configured to obtain activations; and

a plurality of Multiplier-Accumulation (MAC) circuits organized as a plurality of MAC lanes, each of the plurality of MAC lanes configured to:

receive a control signal indicating whether to perform convolution or vector operations;

receive one or more weights from at least one of the plurality of weight selectors according to the control signal;

receive one or more activations from at least one of the plurality of activation input interfaces according to the control signal; and

generate output data based on the one or more weights and the one or more input activations according to the control signal and feed the output data into an output buffer, wherein:

each of the plurality of MAC lanes comprises a plurality of first circuits for performing multiplication operations and a plurality of second circuits organized as a tree, and the second circuits at a leaf level of the tree are configured to receive data from the plurality of first circuits, and

each of the plurality of second circuits comprises an adder-subtractor logic gate configurable to perform addition and subtraction, wherein the control signal comprises a first signal for configuring the adder-subtractor logic gate to perform addition or subtraction, and a second signal for configuring the adder-subtractor logic gate to post-process a result of the addition or subtraction.

2. The vector operation accelerator of claim 1 , wherein each of the plurality of second circuits is configured to:

receive a first input and a second input;

determine whether to perform addition or subtraction based on the first signal;

in response to the first signal indicating performing addition, generate a sum or an average of the first input and the second input based on the second signal; and

in response to the first signal indicating performing subtraction, generate a min or max between the first input and the second input based on the second signal.

3. The vector operation accelerator of claim 2 , wherein the first input and the second input each comprises a vector with a same number of dimensions, and to generate the min between the first input and the second input, each of the plurality of second circuit is further configured to:

generate an output vector comprising min values of the vectors at each corresponding dimension.

4. The vector operation accelerator of claim 1 , further comprising:

a weight matrix generating circuit configured to generate weights for vector reduction operations, wherein the vector reduction operations comprise one or more of reduce mean, reduce minimum, reduce maximum, reduce average, reduce add, or pooling.

5. The vector operation accelerator of claim 4 , wherein each of the plurality of weight selectors comprises a multiplexer coupled with the weight matrix generating circuit and a weight cache.

6. The vector operation accelerator of claim 5 , wherein each of the plurality of weight selectors is configured to:

in response to the control signal indicating performing the convolution computation, obtain a weight from the weight cache; and

in response the control signal indicating performing vector computation, obtain a weight from the weight matrix generating circuit.

7. The vector operation accelerator of claim 1 , further comprising an adder-subtractor circuit outside the tree corresponding to the MAC lane, wherein the adder-subtractor circuit is configured to receive data from the second circuit at a root level of the MAC lane and to write the data into the output buffer.

8. The vector operation accelerator of claim 7 , wherein the adder-subtractor circuit is further configured to:

during a first iteration of computation, write a first set of data received from the second circuit at the root level of the MAC lane into the output buffer; and

during a second iteration of computation:

receive a set of temporary data from the second circuit at the root level of the MAC lane,

retrieve the first set of data from the output buffer,

compute a second set of data based on the set of temporary data, the first set of data, and a control signal indicating whether to perform a convolution computation or a vector operation, and

write the second set of data into the output buffer.

9. The vector operation accelerator of claim 4 , wherein the plurality of MAC lanes are configured to respectively receive a plurality of weight vectors generated by the weight matrix generating circuit for performing a plurality of vector operations in parallel.

10. The vector operation accelerator of claim 4 , wherein a first subset of the plurality of MAC lanes are configured to receive weights from a weight cache and a second subset of the plurality of MAC lanes are configured to receive weights generated by the weight matrix generating circuit, and

the first subset of the plurality of MAC lanes are further configured to perform convolution computations and the second subset of the plurality of MAC lanes are further configured to perform vector operations, and the convolution computations and the vector operations are performed in parallel.

11. A hybrid convolution-vector operation processing system, comprising:

a plurality of weight selectors configured to obtain weights;

a plurality of activation input interfaces configured to obtain activations; and

a plurality of Multiplier-Accumulation (MAC) circuits organized as a plurality of MAC lanes, each of the plurality of MAC lanes configured to:

receive a control signal indicating whether to perform convolution or vector operations;

receive one or more weights from at least one of the plurality of weight selectors according to the control signal;

receive one or more activations from at least one of the plurality of activation input interfaces according to the control signal; and

generate output data based on the one or more weights and the one or more input activations according to the control signal and feed the output data into an output buffer, wherein:

each of the plurality of MAC lanes comprises a plurality of first circuits for performing multiplication operations and a plurality of second circuits organized as a tree, and the second circuits at a leaf level of the tree are configured to receive data from the plurality of first circuits, and

each of the plurality of second circuits comprises an adder-subtractor logic gate configurable to perform addition and subtraction, wherein the control signal comprises a first signal for configuring the adder-subtractor logic gate to perform addition or subtraction, and a second signal for configuring the adder-subtractor logic gate to post-process a result of the addition or subtraction.

12. The hybrid convolution-vector operation processing system of claim 11 , wherein each of the plurality of second circuits is configured to:

receive a first input and a second input;

determine whether to perform addition or subtraction based on the first signal;

in response to the first signal indicating performing addition, generate a sum or an average of the first input and the second input based on the second signal; and

in response to the first signal indicating performing subtraction, generate a min or max between the first input and the second input based on the second signal.

13. The hybrid convolution-vector operation processing system of claim 11 , further comprising:

a weight matrix generating circuit configured to generate weights for vector reduction operations, wherein the vector reduction operations comprise one or more of reduce mean, reduce min, reduce max, reduce average, reduce add, or pooling.

14. The hybrid convolution-vector operation processing system of claim 13 , wherein each of the plurality of weight selectors comprises a multiplexer coupled with the weight matrix generating circuit and a weight cache.

15. The hybrid convolution-vector operation processing system of claim 13 , wherein the plurality of MAC lanes are configured to respectively receive a plurality of weight vectors generated by the weight matrix generating circuit for performing a plurality of vector operations in parallel.

16. The hybrid convolution-vector operation processing system of claim 13 , wherein a first subset of the plurality of MAC lanes are configured to receive weights from a weight cache and a second subset of the plurality of MAC lanes are configured to receive weights generated by the weight matrix generating circuit, and

the first subset of the plurality of MAC lanes are further configured to perform convolution computations and the second subset of the plurality of MAC lanes are further configured to perform vector operations, and the convolution computations and the vector operations are performed in parallel.

17. The hybrid convolution-vector operation processing system of claim 11 , further comprising an adder-subtractor circuit outside the tree corresponding to the MAC lane, wherein the adder-subtractor circuit is configured to receive data from the second at a root level of the MAC lane and to write the data into the output buffer.

18. The hybrid convolution-vector operation processing system of claim 17 , wherein the adder-subtractor circuit is further configured to:

during a first iteration of computation, write a first set of data received from the second at the root level of the MAC lane into the output buffer; and

during a second iteration of computation:

receive a set of temporary data from the second at the root level of the MAC lane,

retrieve the first set of data from the output buffer,

compute a second set of data based on the set of temporary data, the first set of data, and a control signal indicating whether to perform a convolution computation or a vector operation, and

write the second set of data into the output buffer.

19. The hybrid convolution-vector operation processing system of claim 12 , wherein the first input and the second input each comprises a vector with a same number of dimensions, and to generate the min between the first input and the second input, each of the plurality of second circuit is further configured to:

generate an output vector comprising min values of the vectors at each corresponding dimension.

20. The hybrid convolution-vector operation processing system of claim 14 , wherein each of the plurality of weight selectors is configured to:

in response to the control signal indicating performing convolution computation, obtain a weight from the weight cache; and

in response the control signal indicating performing vector computation, obtain a weight from the weight matrix generating circuit.