Compute optimizations for neural networks
One embodiment provides for a compute apparatus to perform machine learning operations, the apparatus comprising a decode unit to decode a single instruction into a decoded instruction that specifies multiple operands including an input value and a quantized weight value associated with a neural network and an arithmetic logic unit including a barrel shifter, an adder, and an accumulator register, wherein to execute the decoded instruction, the barrel shifter is to shift the input value by the quantized weight value to generate a shifted input value and the adder is to add the shifted input value to a value stored in the accumulator register and update the value stored in the accumulator register.
1. A compute apparatus to perform machine learning operations, the apparatus comprising:
a decode unit to decode a single instruction into a decoded instruction that specifies multiple operands including an input value and a quantized weight value associated with a neural network, wherein the quantized weight value is an exponent value for a neural network weight and the neural network weight is constrained to a power of a base value; and
an arithmetic logic unit including a barrel shifter, an adder, and an accumulator register, wherein to execute the decoded instruction, the barrel shifter is to shift the input value by the quantized weight value to generate a shifted input value and the adder is to add the shifted input value to a value stored in the accumulator register and update the value stored in the accumulator register.
2. The compute apparatus as in claim 1 , additionally including an output register to store an output value of the single instruction.
3. The compute apparatus as in claim 1 , wherein the neural network weight is constrained to a power of two value.
4. The compute apparatus as in claim 3 , wherein an exponent associated with the quantized weight value is input to the barrel shifter.
5. The compute apparatus as in claim 4 , wherein the input value is a multiple bit input value.
6. The compute apparatus as in claim 1 , wherein the compute apparatus includes multiple arithmetic logic units configured as a single instruction multiple data compute unit.
7. The compute apparatus as in claim 6 , wherein the single instruction multiple data compute unit is to perform operations for multiple threads of a single instruction multiple thread compute architecture.
8. The compute apparatus as in claim 1 , wherein the compute apparatus is a system on a chip integrated circuit including a media processor and a vision processor.
9. The compute apparatus as in claim 8 , wherein the media processor is to decode multiple simultaneous video streams and output the multiple decoded video streams to an on-chip memory.
10. The compute apparatus as in claim 9 , wherein the vision processor is to parse a decoded video stream to perform processing operations on frames of the decoded video stream via a trained image recognition model associated with the neural network.
11. A method of performing machine learning operations, the method comprising:
decoding a single instruction specifying multiple operands, the operands specifying data including an input value and a quantized weight value of a neural network wherein the quantized weight value is an exponent value for a neural network weight and the neural network weight is constrained to a power of a base value;
issuing the single instruction for execution within a compute unit of a general-purpose graphics processing unit; and
responsive to the execution of the single instruction, generating a result based on shifting the input value by the quantized weight value of the neural network via barrel shifter logic and adding the shifted value to a value stored in an accumulation register.
12. The method as in claim 11 , wherein the neural network weight is constrained to a power of two value.
13. A data processing system comprising:
a general-purpose graphics processing unit comprising a decode unit to decode a single instruction into a decoded instruction that specifies multiple operands including an input value and a quantized weight value associated with a neural network, wherein the quantized weight value is an exponent value for a neural network weight, the neural network weight constrained to a power of a base value, and an arithmetic logic unit including a barrel shifter, an adder, and an accumulator register, wherein to execute the decoded instruction, the barrel shifter is to shift the input value by the quantized weight value to generate a shifted input value and the adder is to add the shifted input value to a value stored in the accumulator register and update the value stored in the accumulator register; and
a memory coupled with the general-purpose graphics processing unit.
14. The data processing system as in claim 13 , the general-purpose graphics processing unit including an output register to store an output value of the single instruction.
15. The data processing system as in claim 13 , wherein the neural network weight is constrained to a power of two value.
16. The data processing system as in claim 15 , wherein an exponent associated with the quantized weight value is input to the barrel shifter.
17. The data processing system as in claim 16 , wherein the input value is a multiple bit input value.