Area-efficient convolutional block
Hardware accelerator designs for neural networks are improved with various approaches to reduce circuit area, improve power consumption, and reduce starvation. Convolutional layers of a neural network may multiply a set of weights with a set of inputs. One example defers two's complement arithmetic from the parallelized multiplication circuits and completes the two's complement arithmetic when the results are accumulated. In another example, a multiplication circuit initially multiplies an input by an initial value of the maximum (or minimum) multiplication range before applying the magnitude of a multiplication encoded relative to the multiplication range. In another example, after dimensional reduction earlier in the network hardware, circuitry for a convolutional layer uses a reduced number of convolutional block circuits that are reused across a plurality of clock cycles to apply different subsets of weight channels.
1 . A circuit for implementing a multiply-and-accumulate function of a neural network, the circuit comprising:
a plurality of pseudo-multiplication circuits configured to compute multiplication products of activations and weights of a convolutional layer of the neural network, a pseudo-multiplication circuit configured to perform a multiplication for the convolutional layer by:
receiving an activation of the convolution layer and an encoded weight, the encoded weight generated from a weight of the convolutional layer,
performing a first shift operation by shifting the activation by a predetermined number of bits, and
performing a second shift operation by shifting a result of the first shift operation based on the encoded weight;
an accumulation circuit coupled to the plurality of pseudo-multiplication circuits and configured to accumulate outputs of the plurality of pseudo-multiplication circuits to generate an accumulation output; and
an adder circuit coupled to the accumulation circuit and configured to add a value to the accumulation output.
2 . The circuit of claim 1 , wherein the value added to the accumulation output is a bias parameter.
3 . The circuit of claim 1 , wherein the value added to the accumulation output is based on a number of negative weights of the convolutional layer.
4 . The circuit of claim 1 , wherein the first shift operation is a left shift operation.
5 . The circuit of claim 1 , wherein the second shift operation is a right shift operation.
6 . The circuit of claim 1 , wherein the encoded weight includes a magnitude indicating a number of bit shifts for the second shift operation.
7 . The circuit of claim 1 , wherein the pseudo-multiplication circuit is configured to perform the multiplication further by:
inverting a result of the second shift operation when the weight is negative.
8 . The circuit of claim 7 , wherein the pseudo-multiplication circuit is to invert the result of the second shift operation by applying an XOR operation on the result of the second shift operation.
9 . The circuit of claim 8 , wherein the pseudo-multiplication circuit comprises a multiplexer, the multiplexer to select a value between zero and a result of the XOR operation, wherein an output of the multiplexer is an output of the pseudo-multiplication circuit.
10 . The circuit of claim 1 , wherein another one of the plurality of pseudo-multiplication circuits is configured to perform a shift operation on another activation of the convolutional layer by the predetermined number of bits.
11 . The circuit of claim 6 , wherein the encoded weight further includes a field indicating a sign of the weight or a field indicating whether the weight is zero.
12 . An apparatus for executing a convolutional layer of a neural network, the apparatus comprising:
one or more pseudo-multiplication units configured to compute multiplication products of activations and weights of the convolutional layer, a pseudo-multiplication unit configured to perform a multiplication for the convolutional layer by:
receiving an activation of the convolution layer and an encoded weight, the encoded weight generated from a weight of the convolutional layer,
performing a first shift operation by shifting the activation by a predetermined number of bits, and
performing a second shift operation by shifting a result of the first shift operation based on the encoded weight; and
an accumulation unit coupled to the one or more pseudo-multiplication units and configured to accumulate outputs of the one or more pseudo-multiplication units to generate an accumulation output.
13 . The apparatus of claim 12 , wherein the first shift operation is a left shift operation.
14 . The apparatus of claim 12 , wherein the second shift operation is a right shift operation.
15 . The apparatus of claim 12 , wherein the encoded weight includes a magnitude indicating a number of bit shifts for the second shift operation.
16 . The apparatus of claim 15 , wherein the encoded weight further includes a field indicating a sign of the weight or a field indicating whether the weight is zero.
17 . The apparatus of claim 12 , wherein the pseudo-multiplication unit is configured to perform the multiplication further by:
inverting a result of the second shift operation when the weight is negative.
18 . The apparatus of claim 17 , wherein the pseudo-multiplication unit is to invert the result of the second shift operation by applying an XOR operation on the result of the second shift operation.
19 . The apparatus of claim 18 , wherein the pseudo-multiplication unit comprises a multiplexer, the multiplexer to select a value between zero and a result of the XOR operation, wherein an output of the multiplexer is an output of the pseudo-multiplication unit.
20 . The apparatus of claim 12 , wherein another one of the plurality of pseudo-multiplication units is configured to perform a shift operation on another activation of the convolutional layer by the predetermined number of bits.
21 . An apparatus for executing multiplications of activations and weights for a convolutional layer of a neural network, the apparatus comprising:
a first shift operator to receive an activation of the convolutional layer and to shift the activation by a predetermined number of bits;
a second shift operator to receive an encoded weight generated from a weight of the convolutional layer and to shift an output of the first shift operator based on the encoded weight;
an XOR gate to invert a sign of an output of the second shift operator; and
a multiplexer to select a value between an output of the XOR gate and zero.
22 . The apparatus of claim 21 , wherein the first shift operator is configured to perform left shift, and the second shift operator is configured to perform right shift.
23 . The apparatus of claim 21 , wherein the encoded weight includes a field indicating a number of bit shifts, wherein the second shift operator is to shift the output of the first shift operator based on the field of the encoded weight.
24 . The apparatus of claim 21 , wherein the encoded weight includes a field indicating a sign of the weight, wherein the XOR gate to invert the sign of the output of the second shift operator based on the field of the encoded weight.
25 . The apparatus of claim 21 , wherein the encoded weight includes a field indicating whether the weight is zero, wherein the multiplexer is to select the value based on the field of the encoded weight.