Compute optimization mechanism
An apparatus to facilitate compute optimization is disclosed. The apparatus includes a mixed precision core to perform a mixed precision multi-dimensional matrix multiply and accumulate operation on 8-bit and/or 32 bit signed or unsigned integer elements.
1. A graphics processor comprising:
a memory controller;
a level-two (L2) cache memory coupled with the memory controller; and
a multiprocessor coupled to the memory controller, the multiprocessor having a single instruction, multiple thread (SIMT) architecture including hardware multithreading, the multiprocessor including a scheduler, a plurality of processing cores, and a shared memory coupled to the plurality of processing cores, wherein the scheduler is a hardware scheduler to schedule an instruction for execution by the plurality of processing cores and the plurality of processing cores include a mixed precision core to perform a mixed precision matrix multiply and accumulate operation in response to the instruction, wherein to perform the mixed precision matrix multiply and accumulate operation, the mixed precision core is to perform an operation D=A*B+C, wherein A, B, C, and D are matrix elements, A and B are 8-bit integer elements, and C is a 32-bit integer element, and wherein the instruction is to specify a first format for matrix elements A and B, separately specify a second format for matrix element C, and the mixed precision core is to perform the operation according to the specified formats.
2. The graphics processor as in claim 1 , the multiprocessor additionally including a register file to store data associated with operands, wherein the instruction is additionally to specify a separate format for matrix element D.
3. The graphics processor as in claim 2 , wherein the multiprocessor is to load the data associated with operands of the operation D=A*B+C into the register file from memory.
4. The graphics processor as in claim 3 , wherein the multiprocessor is to load the data associated with operands of the operation D=A*B+C into the register file from shared memory.
5. The graphics processor as in claim 4 , wherein the multiprocessor is to load the data associated with operands of the operation D=A*B+C into the register file from the L2 cache memory in response to the load from the shared memory.
6. The graphics processor of claim 1 , wherein the multiprocessor includes an instruction cache to store the instruction for execution at the multiprocessor, wherein the instruction is executed as a warp of threads, and the mixed precision core is to execute multiple threads of the warp of threads.
7. The graphics processor of claim 1 , wherein A and B are capable of being selected as either a signed 8-bit integer element or an unsigned 8-bit integer element and C is a signed 32-bit integer element.
8. The graphics processor of claim 1 , wherein the hardware scheduler is implemented via a microcontroller.
9. A graphics processing system comprising:
a graphics memory device;
a memory controller coupled to the graphics memory device;
a level-two (L2) cache memory coupled with the memory controller and the graphics memory device; and
a multiprocessor coupled to the memory controller, the multiprocessor having a single instruction, multiple thread (SIMT) architecture including hardware multithreading, the multiprocessor including a scheduler, a plurality of processing cores, and a shared memory coupled to the plurality of processing cores, wherein the scheduler is a hardware scheduler to schedule an instruction for execution by the plurality of processing cores and the plurality of processing cores include a mixed precision core to perform a mixed precision matrix multiply and accumulate operation in response to the instruction, wherein to perform the mixed precision matrix multiply and accumulate operation, the mixed precision core is to perform an operation D=A*B+C, wherein A, B, C, and D are matrix elements, A and B are 8-bit integer elements, and C is a 32-bit integer element, and wherein the instruction is to specify a first format for matrix elements A and B, separately specify a second format for matrix element C, and the mixed precision core is to perform the operation according to the specified formats.
10. The graphics processing system of claim 9 , wherein the graphics memory device includes graphics double data rate (GDDR) memory.
11. The graphics processing system of claim 10 , wherein the GDDR memory includes GDDR6 memory.
12. The graphics processing system as in claim 9 , wherein the multiprocessor additionally includes a register file to store data associated with operands of the operation D=A*B+C and the multiprocessor is to load the data associated with the operands into the register file from the graphics memory device and the instruction is additionally to specify a separate format for matrix element D.
13. The graphics processing system as in claim 9 , wherein the multiprocessor additionally includes a register file to store data associated with operands of the operation D=A*B+C and the multiprocessor is to load the data associated with the operands into the register file from the L2 cache memory.
14. The graphics processing system of claim 9 , wherein A and B each include values associated with a layer of a neural network.
15. The graphics processing system of claim 14 , wherein A or B each include values associated with a weight of the layer of the neural network or an input to the layer of the neural network.
16. The graphics processing system of claim 9 , wherein A and B are capable of being selected as either a signed 8-bit integer element or an unsigned 8-bit integer element and C is a signed 32-bit integer element.
17. The graphics processing system of claim 9 , wherein the hardware scheduler is implemented via a microcontroller.
18. A method comprising:
decoding an instruction into a decoded instruction for execution by a graphics processor including a multiprocessor coupled to a memory controller and a level-two (L2) cache memory coupled with the memory controller, the multiprocessor having a single instruction, multiple thread (SIMT) architecture including hardware multithreading, wherein the multiprocessor includes a scheduler, a plurality of processing cores, and a shared memory coupled to the plurality of processing cores, and wherein the plurality of processing cores include a mixed precision core to perform a mixed precision matrix multiply and accumulate operation;
scheduling the decoded instruction for execution by the multiprocessor; and
executing the decoded instruction via the mixed precision core, wherein the mixed precision core, in response to the instruction is to perform the mixed precision matrix multiply and accumulate operation and wherein performing the matrix multiply and accumulate operation includes performing an operation D=A*B+C, wherein A, B, C, and D are matrix elements, A and B are 8-bit integer elements, and C is a 32-bit integer element, and wherein the instruction specifies a first format for matrix elements A and B, separately specifies a second format for matrix element C, and the mixed precision core performs the operation according to the specified formats.
19. The method as in claim 18 , further comprising fetching the instruction from an instruction cache and dispatching a warp of threads associated with the instruction for execution by the mixed precision core, and the mixed precision core is to execute multiple threads of the warp of threads.
20. The method as in claim 18 , wherein A or B each include values associated with a weight of a layer of a neural network or an input to the layer of the neural network and A and B are capable of being selected as either a signed 8-bit integer element or an unsigned 8-bit integer element and C is a signed 32-bit integer element.