Multiply-accumulate “0” data gating
In an example, an apparatus comprises a plurality of execution units and logic, at least partially including hardware logic, to gate at least one of a multiply unit or an accumulate unit in response to an input of value zero. Other embodiments are also disclosed and claimed.
1. A general purpose graphics processor, comprising:
a plurality of execution units comprising at least a first type of execution unit having a first set of execution resources and a second type of execution unit having a second set of execution resources, different from the first set of execution resources; and
processing circuitry to:
receive a first inference weight and a second inference weight from a layer of a convolutional neural network into at least one of a multiply unit or an accumulate unit; and
bypass the at least one of a multiply unit or an accumulate unit, such that the at least one of multiply unit or an accumulate unit performs no operations on the first inference weight or the second inference weight, in response to at least one of the first inference weight or the second inference weigh having a value zero.
2. The general purpose graphics processor of claim 1 , wherein the multiply unit comprises a 32 bit multiply and accumulate unit to:
receive the first inference weight and a first indicator of a number of valid bits in the first inference weight;
receive the second inference weight, and a second indicator of a number of valid bits in the second inference weight; and
multiply only the valid bits in the first inference weight and valid bits in the second inference weight.
3. The apparatus of claim 2 , further comprising processing circuitry to:
produce no output when at least one of the first operand or the second operand is zero.
4. The general purpose graphics processor of claim 2 , further comprising processing circuitry to:
bypass the multiply unit when at least one of the first operand or the second operand is one.
5. The general purpose graphics processor of claim 2 , further comprising processing circuitry to:
negate the second operand and bypass the multiplier when at least one of the first operand or the second operand is a negative one.
6. The general purpose graphics processor of claim 2 , further comprising processing circuitry to:
bypass the multiply unit and use a shift register when at least one of the first operand or the second operand is a power of two.
7. The general purpose graphics processor of claim 2 , further comprising processing circuitry to:
treat at least one of the first operand or the second operand as a zero when a value of the first operand or the second operand is within a threshold of zero.
8. The general purpose graphics processor of claim 2 , further comprising processing circuitry to:
treat at least one of the first operand or the second operand as a power of two when a value of the first operand or the second operand is within a threshold of a power of two.
9. The general purpose graphics processor of claim 2 , further comprising a thread scheduler comprising logic, at least partially including hardware logic, to:
break an input vector into a plurality of segments; and
perform a dot product using the plurality of segments.
10. The general purpose graphics processor of claim 1 , wherein the plurality of execution units are on a single integrated circuit.
11. An electronic device, comprising:
a processor having a plurality of execution units comprising at least a first type of execution unit having a first set of execution resources and a second type of execution unit having a second set of execution resources, different from the first set of execution resources; and
processing circuitry to:
receive a first inference weight and a second inference weight from a layer of a convolutional neural network into at least one of a multiply unit or an accumulate unit; and
bypass the at least one of a multiply unit or an accumulate unit, such that the at least one of multiply unit or an accumulate unit performs no operations on the first inference weight or the second inference weight, in response to at least one of the first inference weight or the second inference weigh having a value zero.
12. The electronic device of claim 11 , wherein the multiply unit comprises a 32 bit multiply and accumulate unit to:
receive the first inference weight and a first indicator of a number of valid bits in the first weight;
receive the second inference weight, and a second indicator of a number of valid bits in the second weight; and
multiply only the valid bits in the first inference weight and valid bits in the second inference weight.
13. The electronic device of claim 12 , further comprising processing circuitry to:
produce no output when at least one of the first operand or the second operand is zero.
14. The electronic device of claim 12 , further comprising processing circuitry to:
bypass the multiply unit when at least one of the first operand or the second operand is one.
15. The electronic device of claim 12 , further comprising processing circuitry to:
negate the second operand and bypass the multiplier when at least one of the first operand or the second operand is a negative one.
16. The electronic device of claim 12 , further comprising processing circuitry to:
bypass the multiply unit and use a shift register when at least one of the first operand or the second operand is a power of two.
17. The electronic device of claim 12 , further processing circuitry to:
treat at least one of the first operand or the second operand as a zero when a value of the first operand or the second operand is within a threshold of zero.
18. The electronic device of claim 12 , further comprising a driver comprising processing circuitry to:
treat at least one of the first operand or the second operand as a power of two when a value of the first operand or the second operand is within a threshold of a power of two.
19. The electronic device of claim 12 , further comprising a thread scheduler comprising processing circuitry to:
break an input vector into a plurality of segments; and
perform a dot product using the plurality of segments.
20. The electronic device of claim 11 , wherein the plurality of execution units are on a single integrated circuit.