Compute optimization mechanism for deep neural networks
An apparatus to facilitate compute optimization is disclosed. The apparatus includes a plurality of processing units each comprising a plurality of processing cores of a first type and a second type. A first set of processing cores of a first type perform multi-dimensional matrix operations and a second set of processing cores of a second type perform general purpose graphics processing unit (GPGPU) operations.
1. A graphics processing unit comprising one or more multiprocessors, at least one of the one or more multiprocessors including:
a register file to store a plurality of different types of operands; and
a plurality of processing cores, including:
a first set of processing cores of a first type to perform multi-dimensional matrix operations on a first set of operands in a first set of registers of the register file, wherein the first set of processing cores of the first type includes circuitry to execute instructions to perform matrix operations on the first set of operands in the first set of registers of the register file, the first set of operands including one or more 64-bit operands; and
a second set of processing cores of a second type, the second set of processing cores being different from the first set of processing cores, the second set of processing cores to perform general purpose graphics processing unit (GPGPU) operations on a second set of operands in a second set of registers of the register file, wherein the second set of processing cores comprises:
a set of floating point units (FPUs) to execute instructions to perform floating point operations, the set of FPUs to perform 32-bit floating point (FP32) operations and 16-bit floating point (FP16) operations; and
a set of integer units to execute instructions to perform integer operations.
2. The graphics processing unit as in claim 1 , wherein the first set of processing cores of the first type is configured to perform an in-place matrix to vector transformation for a first type of operand stored in a shared local memory associated with the at least one of the one or more multiprocessors.
3. The graphics processing unit as in claim 1 , wherein the set of FPUs includes first FPUs to perform the FP32 operations and second FPUs to perform the FP16 operations.
4. The graphics processing unit as in claim 1 , wherein the first set of processing cores of the first type is configured to perform an in-place matrix to vector transformation for a first type of operand stored in the register file.
5. The graphics processing unit as in claim 4 , wherein the in-place matrix to vector transformation includes a set of operations having a source and destination, the source and destination within the register file.
6. The graphics processing unit as in claim 5 , wherein the source includes a register address start limit, stride, number of elements, and element size.
7. The graphics processing unit as in claim 1 , wherein the at least one of the one or more multiprocessors further comprises an instruction cache to store a first instruction associated with the first set of operands and a second instruction associated with the second set of operands.
8. The graphics processing unit as in claim 1 , wherein the first set of processing cores of the first type are associated with a first memory channel and the second set of processing cores of the second type are associated with a second memory channel.
9. The graphics processing unit as in claim 1 , wherein the one or more multiprocessors have a single instruction multiple thread (SIMT) architecture.
10. A method to facilitate processing of data at a graphics processing unit (GPU), the method comprising:
receiving, at a first set of processing cores of a first type, a first set of operands from first registers of a register file, wherein the first set of processing cores of the first type includes circuitry to execute instructions to perform matrix operations on the first set of operands in the first set of registers of the register file, the first set of operands including one or more 64-bit operands;
receiving, at a second set of processing cores of a second type, a second set of operands from second registers of the register file, the second set of processing cores being different from the first set of processing cores;
performing multi-dimensional matrix math operations on the first set of operands at the first set of processing cores; and
performing general-purpose graphics processing unit (GPGPU) operations on the second set of operands at the second set of processing cores, wherein performing the GPGPU operations at the second set of processing cores comprises:
executing instructions at a set of floating point units (FPUs) to perform floating point operations, wherein executing the instructions at the set of FPUs comprises performing 32-bit floating point (FP32) operations and 16-bit floating point (FP16) operations; and
executing instructions at a set of integer units to perform integer operations.
11. The method as in claim 10 , further comprising performing an in-place matrix to vector transformation for a first type of operand stored in a shared local memory associated with the first set of processing cores of the first type.
12. The method as in claim 10 , further comprising performing the FP32 operations at first FPUs of the set of FPUs and performing the FP16 operations at second FPUs of the set of FPUs.
13. The method as in claim 10 , further comprising performing an in-place matrix to vector transformation for a first type of operand stored in the register file via the first set of processing cores of the first type, wherein the in-place matrix to vector transformation includes a set of operations having a source and destination and the source and destination are within the register file.
14. The method as in claim 13 , wherein the source includes a register address start limit, stride, number of elements, and element size.
15. The method as in claim 10 , wherein the GPU includes one or more multiprocessors comprising the first set of processing cores of the first type, the second set of processing cores of the second type, and an instruction cache to store a first instruction associated with the first set of operands and a second instruction associated with the second set of operands.
16. The method as in claim 15 , wherein the first set of processing cores of the first type are associated with a first memory channel and the second set of processing cores of the second type are associated with a second memory channel.
17. The method as in claim 16 , wherein the one or more multiprocessors have a single instruction multiple thread (SIMT) architecture.
18. A graphics processing system comprising:
a graphics processing unit comprising one or more multiprocessors, at least one of the one or more multiprocessors including a register file to store a plurality of different types of operands and a plurality of processing cores, including:
a first set of processing cores of a first type to perform multi-dimensional matrix operations on a first set of operands in a first set of registers of the register file, wherein the first set of processing cores of the first type includes circuitry to execute instructions to perform matrix operations on the first set of operands in the first set of registers of the register file; and
a second set of processing cores of a second type, the second set of processing cores being different from the first set of processing cores, the second set of processing cores to perform general purpose graphics processing unit (GPGPU) operations on a second set of operands in a second set of registers of the register file, wherein the second set of processing cores comprises:
a set of floating point units (FPUs) to execute instructions to perform floating point operations, the set of FPUs to perform 32-bit floating point (FP32) operations and 16-bit floating point (FP16) operations; and
a set of integer units to execute instructions to perform integer operations.
19. The graphics processing system as in claim 18 , wherein the first set of operands include one or more 64-bit operands.
20. The graphics processing system as in claim 18 , wherein the set of FPUs includes first FPUs to perform the FP32 operations and second FPUs to perform the FP16 operations.
21. The graphics processing system as in claim 18 , wherein the first set of processing cores of the first type is configured to perform an in-place matrix to vector transformation for a first type of operand stored in the register file.
22. The graphics processing system as in claim 21 , wherein the in-place matrix to vector transformation includes a set of operations having a source and destination, the source and destination within the register file.
23. The graphics processing system as in claim 22 , wherein the source includes a register address start limit, stride, number of elements, and element size.
24. The graphics processing system as in claim 18 , wherein the at least one of the one or more multiprocessors further comprises an instruction cache to store a first instruction associated with the first set of operands and a second instruction associated with the second set of operands.
25. The graphics processing system as in claim 18 , wherein the first set of processing cores of the first type are associated with a first memory channel and the second set of processing cores of the second type are associated with a second memory channel.