Bitwise product-sum accumulations with skip logic
A method, device, and system for performing a partial sum accumulation of a product of input vectors and weight vectors in a wordwise-input and bitwise-weight manner results in a partial accumulated product sum. The partial accumulated product sum is compared with a threshold condition after each weight bit, and when the partial accumulated product sum meets the threshold condition, a skip indicator is asserted to indicate that remaining computations of a sum accumulation are skipped.
1 . A device comprising:
a multiply accumulate circuit block configured to respectively multiply a set of M input vectors by first weight bits of a set of M weight vectors to calculate a first set of M products, add the first set of M products together to calculate a first product sum, and provide the first product sum on a product sum output pin set;
an accumulator circuit block comprising an accumulator input pin set coupled to the product sum output pin set, the accumulator circuit block configured to receive a first value on the accumulator input pin set, add the first value to an accumulated value stored in the accumulator circuit block to generate an updated accumulated value, replace the accumulated value with the updated accumulated value, and provide the updated accumulated value on an accumulated value output pin set;
a skip circuit block comprising an accumulated value input pin set coupled to the accumulated value output pin set, the skip circuit block configured to receive the updated accumulated value on the accumulated value output pin set, compare the updated accumulated value to a threshold value to detect a skip condition, and in response to detection of the skip condition, assert a skip signal on a skip pin, wherein the skip circuit block comprises a subtraction circuit block configured to subtract a shifted input sum from an input sum to calculate the threshold value, the shifted input sum shifted a number of times based on a weight bit position of the set of M weight vectors being processed; and
a control circuit block configured to check the skip pin for the skip signal and in response to detecting that the skip signal is asserted: terminate processing remaining weight bits of the set of M weight vectors, assert an output ready signal, and provide the updated accumulated value to an activation circuit.
2 . The device of claim 1 , wherein the threshold value is a first threshold value, wherein the subtraction circuit block is further configured to alter the updated accumulated value by adding a second threshold value to the updated accumulated value prior to comparing the first threshold value to the updated accumulated value.
3 . The device of claim 2 , wherein the second threshold value is a constant value, a derived value based on the input sum or the updated accumulated value, or a dynamic value based on an environmental variable, the environmental variable including a power status, battery status, hardware status, or resource status.
4 . The device of claim 1 , wherein the accumulator circuit block comprises an ADD pin, wherein the accumulator circuit block is configured to add the first value to the accumulated value when a control signal at the ADD pin has a first logic value, wherein the accumulator circuit block is configured to subtract the first value from the accumulated value when the control signal at the ADD pin has a second logic value.
5 . The device of claim 4 , wherein the control circuit block is further configured to set the control signal at the ADD pin to the second logic value when the first weight bits are most significant bits (MSBs) of the set of M weight vectors.
6 . The device of claim 5 , wherein the control circuit block is further configured to set the control signal at the ADD pin to the first logic value when the first weight bits are not MSBs of the set of M weight vectors.
7 . The device of claim 1 , further comprising:
an input latch circuit block configured to store the set of M input vectors; and
a weight bit latch circuit block configured to store the set of M weight vectors,
wherein the multiply accumulate circuit block comprises a multiplier and an adder tree to calculate the first product sum.
8 . The device of claim 1 , wherein the skip circuit block is configured to detect the skip condition by:
comparing an absolute value of the updated accumulated value with the threshold value; and
in response to detecting that the absolute value of the updated accumulated value is larger than the threshold value, declaring that the skip condition is detected.
9 . The device of claim 8 , wherein the control circuit block is further configured to, in response to detecting that the skip signal is not asserted, instruct the multiply accumulate circuit block to process second weight bits of the set of M weight vectors next to the first weight bits.
10 . The device of claim 8 , further comprising the activation circuit, wherein the activation circuit is configured to:
output a zero value when the updated accumulated value is less than or equal to zero; and
output the updated accumulated value when the updated accumulated value is larger than zero.
11 . The device of claim 1 , wherein the input sum is a sum of the set of M input vectors.
12 . A device comprising:
a multiply accumulate circuit configured to:
receive a set of M input vectors and a set of M weight vectors, wherein each input vector in the set of M input vectors has a corresponding weight vector in the set of M weight vectors;
calculate a first set of M products by performing an element-wise multiplication between the set of M input vectors and a first bit vector of length M, wherein the first bit vector comprises weight bits at a first bit location of the set of M weight vectors;
add the first set of M products together to calculate a first product sum; and
provide the first product sum to an output terminal of the multiply accumulate circuit;
an accumulator circuit, wherein an input terminal of the accumulator circuit is coupled to the output terminal of the multiply accumulate circuit, wherein the accumulator circuit is configured to:
subtract a first value at the input terminal of the accumulator circuit from an accumulated value stored in the accumulator circuit to generate an updated accumulated value, when the first bit location is a most significant bit (MSB) location of the set of M weight vectors;
add the first value at the input terminal of the accumulator circuit to the accumulated value stored in the accumulator circuit to generate the updated accumulated value, when the first bit location is not the MSB location of the set of M weight vectors;
replace the accumulated value stored in the accumulator circuit with the updated accumulated value; and
provide the updated accumulated value to an output terminal of the accumulator circuit;
a skip circuit, wherein an input terminal of the skip circuit is coupled to the output terminal of the accumulator circuit, wherein the skip circuit is configured to:
receive the updated accumulated value from the output terminal of the accumulator circuit;
compare the updated accumulated value to a threshold value; and
in response to detecting that an absolute value of the updated accumulated value is larger than the threshold value, assert a skip signal on an output pin of the skip circuit, wherein the skip circuit is configured to generate the threshold value for weight bits at non-MSB bit locations of the set of M weight vectors by:
shifting a value stored in the skip circuit to the left by one bit to generate a bit-shifted value; and
subtracting the bit-shifted value from an input sum, wherein the input sum is a sum of the set of M input vectors; and
a control circuit coupled to the multiply accumulate circuit, the accumulator circuit, and the skip circuit, wherein the control circuit is configured to:
check the skip signal on the output pin of the skip circuit; and
in response to detecting that the skip signal is asserted, terminate processing for remaining weight bits of the set of M weight vectors and assert an output ready signal.
13 . The device of claim 12 , wherein the control circuit is configured to initialize the accumulated value stored in the accumulator circuit to zero.
14 . The device of claim 13 , wherein the control circuit is further configured to, in response to detecting that the skip signal is not asserted, instruct the multiply accumulate circuit to:
calculate a second set of M products by performing an element-wise multiplication between the set of M input vectors and a second bit vector of length M, wherein the second bit vector comprises weight bits at a second bit location of the set of M weight vectors, wherein the second bit location is adjacent to the first bit location;
add the second set of M products together to calculate a second product sum; and
provide the second product sum to the output terminal of the multiply accumulate circuit.
15 . The device of claim 14 , wherein the control circuit is further configured to, in response to detecting that the skip signal is not asserted, instruct the accumulator circuit to:
add the first value at the input terminal of the accumulator circuit to the accumulated value stored in the accumulator circuit to generate the updated accumulated value;
replace the accumulated value stored in the accumulator circuit with the updated accumulated value; and
provide the updated accumulated value to the output terminal of the accumulator circuit.
16 . The device of claim 12 , wherein an initial value for the value stored in the skip circuit is equal to the input sum shifted to the right by K−1 bits, wherein K is the number of bits in each of the M weight vectors.
17 . A device comprising:
a multiply accumulate circuit comprising:
a multiplier configured to calculate a set of M products by:
receiving a set of M input vectors and a set of M weight vectors, wherein each input vector in the set of M input vectors has a corresponding weight vector in the set of M weight vectors; and
performing an element-wise multiplication between the set of M input vectors and a bit vector of length M, wherein the bit vector comprises weight bits at a first bit location of the set of M weight vectors; and
an adder tree configured to add the set of M products together to calculate a product sum;
an accumulator circuit comprising an accumulator and a shift register, wherein a first input terminal of the accumulator is coupled to an output terminal of the adder tree, a second input terminal of the accumulator is coupled to an output terminal of the shift register, and an input terminal of the shift register is coupled to an output terminal of the accumulator;
a skip circuit, wherein an input terminal of the skip circuit is coupled to the output terminal of the accumulator, wherein the skip circuit is configured to:
receive a value from the output terminal of the accumulator;
compare the value to a threshold value; and
in response to detecting that a magnitude of the value is larger than the threshold value, assert a skip signal on an output pin of the skip circuit, wherein the skip circuit is configured to calculate the threshold value by subtracting a shifted input sum from an input sum, wherein the shifted input sum is the input sum shifted by a number of bits based on the first bit location of the weight bits being processed; and
a control circuit coupled to the multiply accumulate circuit, the accumulator circuit, and the skip circuit, wherein the control circuit is configured to:
check the skip signal on the output pin of the skip circuit; and
in response to detecting that the skip signal is asserted, terminate processing for remaining weight bits of the set of M weight vectors and assert an output ready signal.
18 . The device of claim 17 , wherein the accumulator circuit is configured to:
subtract a first value at the first input terminal of the accumulator from an accumulated value stored in the shift register to generate an updated accumulated value at the output terminal of the accumulator, when the first bit location is a most significant bit (MSB) location of the set of M weight vectors;
add the first value at the first input terminal of the accumulator to the accumulated value stored in the shift register to generate the updated accumulated value at the output terminal of the accumulator, when the first bit location is not the MSB location of the set of M weight vectors; and
replace the accumulated value stored in the shift register with the updated accumulated value.
19 . The device of claim 17 , wherein the input sum is a sum of the set of M input vectors.
20 . The device of claim 17 , wherein the skip circuit is further configured to, before comparing the value to the threshold value, adjust the value by adding an adjustment value to the value.