Accelerate neural networks with compression at different levels
A neural network accelerator includes 2 n multiplier circuits, 2 n shifter circuits and an adder tree circuit. Each respective multiplier circuit multiplies a first value by a second value to output a first product value. Each respective first value is represented by a first predetermined number of bits beginning at a most significant bit of the first value having a value equal to 1. Each respective second value is represented by a second predetermined number of bits, and each respective first product value is represented by a third predetermined number of bits. Each respective shifter circuit receives the first product value of a corresponding multiplier circuit and left shifts the corresponding product value by the first predetermined number of bits to form a respective second product value. The adder circuit adds each respective second product value to form a partial-sum value represented by a fourth predetermined number of bits.
1 . A neural network accelerator, comprising:
2 n multiplier circuits, each respective multiplier circuit being configured to multiply a corresponding first value by a corresponding second value to output a corresponding first product value in which n comprises an integer, each respective first value being represented by a first predetermined number of bits beginning at a most significant bit of the first value having a value equal to 1, each respective second value being represented by a second predetermined number of bits, and each respective first product value being represented by a third predetermined number of bits;
2 n shifter circuits, each respective shifter circuit being configured to receive the first product value of a corresponding multiplier circuit and to left shift the first corresponding product value by the first predetermined number of bits to form a respective second product value; and
an adder circuit configured to add each respective second product value to form a partial-sum value represented by a fourth predetermined number of bits.
2 . The neural network accelerator of claim 1 , wherein each respective second value being further represented by the second predetermined number of bits beginning at a most significant bit of the second value having a value equal to 1, and
wherein each respective shifter circuit further configured to left shift the first corresponding product value by a first number equal to the first predetermined number of bits plus a second number equal to the second predetermined number of bits to form the respective second product value.
3 . The neural network accelerator of claim 2 , wherein n equals 5, the first predetermined number of bits equals 4 bits, the second predetermined number of bits equals 4 bits, the third predetermined number of bits equals 8 bits, and the fourth predetermined number of bits equals 21 bits.
4 . The neural network accelerator of claim 1 , wherein n equals 5, the first predetermined number of bits equals 4 bits, the second predetermined number of bits equals 8 bits, the third predetermined number of bits equals 12 bits, and the fourth predetermined number of bits equals 21 bits.
5 . The neural network accelerator of claim 1 , further comprising an accumulator configured to accumulate the partial-sum value with other partial-sum values to form a final sum value.
6 . The neural network accelerator of claim 1 , wherein the adder circuit comprises an adder tree circuit.
7 . The neural network accelerator of claim 1 , wherein the corresponding first value comprises an activation value and the corresponding second value comprises a weight value.
8 . The neural network accelerator of claim 1 , wherein at least one of the corresponding first value or the corresponding second value comprises a truncated value.
9 . The neural network accelerator of claim 1 , wherein the corresponding first product value comprises a truncated value.
10 . A method comprising:
outputting, by each of 2 n multiplier circuits wherein n comprises an integer, a corresponding first product value, each respective multiplier circuit being configured to multiply a corresponding first value by a corresponding second value to output the corresponding first product value, each respective first value being represented by a first predetermined number of bits beginning at a most significant bit of the first value having a value equal to 1, each respective second value being represented by a second predetermined number of bits, and each respective first product value being represented by a third predetermined number of bits;
receiving, by each of 2 n shifter circuits, the first product value of a corresponding multiplier circuit, each respective shifter circuit being configured to left shift the first corresponding product value by the first predetermined number of bits to form a respective second product value; and
forming, by an adder circuit configured to add each respective second product value, a partial-sum value represented by a fourth predetermined number of bits.
11 . The method of claim 10 , wherein each respective second value being further represented by the second predetermined number of bits beginning at a most significant bit of the second value having a value equal to 1, and
wherein each respective shifter circuit further configured to left shift the first corresponding product value by a first number equal to the first predetermined number of bits plus a second number equal to the second predetermined number of bits to form the respective second product value.
12 . The method of claim 11 , wherein n equals 5, the first predetermined number of bits equals 4 bits, the second predetermined number of bits equals 4 bits, the third predetermined number of bits equals 8 bits, and the fourth predetermined number of bits equals 21 bits.
13 . The method of claim 10 , further comprising:
accumulating the partial-sum value with other partial-sum values; and
forming a final sum value based on accumulating the partial-sum value with the other partial-sum values.
14 . The method of claim 10 , wherein the adder circuit comprises an adder tree circuit.
15 . The method of claim 10 , wherein the corresponding first value comprises an activation value and the corresponding second value comprises a weight value.
16 . The method of claim 10 , wherein at least one of the corresponding first value or the corresponding second value comprises a truncated value.
17 . The method of claim 10 , wherein the corresponding first product value comprises a truncated value.
18 . A system comprising:
a neural network; and
a neural network accelerator comprising:
2 n multiplier circuits, each respective multiplier circuit being configured to multiply a corresponding first value by a corresponding second value to output a corresponding first product value in which n comprises an integer, each respective first value being represented by a first predetermined number of bits beginning at a most significant bit of the first value having a value equal to 1, each respective second value being represented by a second predetermined number of bits, and each respective first product value being represented by a third predetermined number of bits;
2 n shifter circuits, each respective shifter circuit being configured to receive the first product value of a corresponding multiplier circuit and to left shift the first corresponding product value by the first predetermined number of bits to form a respective second product value; and
an adder circuit configured to add each respective second product value to form a partial-sum value represented by a fourth predetermined number of bits.
19 . The system of claim 18 , wherein each respective second value being further represented by the second predetermined number of bits beginning at a most significant bit of the second value having a value equal to 1, and
wherein each respective shifter circuit further configured to left shift the first corresponding product value by a first number equal to the first predetermined number of bits plus a second number equal to the second predetermined number of bits to form the respective second product value.
20 . The system of claim 19 , wherein n equals 5, the first predetermined number of bits equals 4 bits, the second predetermined number of bits equals 4 bits, the third predetermined number of bits equals 8 bits, and the fourth predetermined number of bits equals 21 bits.