Instructions and logic to perform floating-point and integer operations for machine learning
One embodiment provides for a graphics processing unit to accelerate machine-learning operations, the graphics processing unit comprising a multiprocessor having a single instruction, multiple thread (SIMT) architecture, the multiprocessor to execute at least one single instruction; and a first compute unit included within the multiprocessor, the at least one single instruction to cause the first compute unit to perform a two-dimensional matrix multiply and accumulate operation, wherein to perform the two-dimensional matrix multiply and accumulate operation includes to compute a 32-bit intermediate product of 16-bit operands and to compute a 32-bit sum based on the 32-bit intermediate product.
1. A graphics processing unit to accelerate machine-learning operations, the graphics processing unit comprising:
a multiprocessor having a single instruction, multiple thread (SIMT) architecture, the multiprocessor to execute at least one single instruction across multiple threads of the multiprocessor; and
a first compute unit included within the multiprocessor, the at least one single instruction to cause the first compute unit to perform a two-dimensional matrix multiply and accumulate operation, wherein to perform the two-dimensional matrix multiply and accumulate operation includes to compute an intermediate product of 16-bit operands and to compute a 32-bit sum based on the intermediate product;
wherein to compute a 32-bit sum based on the intermediate product, the first compute unit is to:
perform a floating-point multiply of two or more 16-bit operands at a first intermediate precision to generate the intermediate product, the first intermediate precision less than 32 bits;
compute a sum based on the intermediate product to generate an intermediate sum at a second intermediate precision; and
compute the 32-bit sum via a conversion of the intermediate sum at the second intermediate precision to a 32-bit precision.
2. The graphics processing unit as in claim 1 , the multiprocessor to execute parallel threads of a thread group, each thread of the thread group having independent thread state.
3. The graphics processing unit as in claim 2 , the multiprocessor including a scheduler to schedule the parallel threads of the thread group to multiple compute units within the multiprocessor.
4. The graphics processing unit as in claim 3 , the multiple compute units within the multiprocessor including a second compute unit to perform an integer operation, the scheduler to schedule a floating-point operation to the first compute unit and an integer operation to the second compute unit wherein the multiprocessor is to concurrently execute a floating-point operation on the first compute unit and an integer operation on the second compute unit.
5. The graphics processing unit as in claim 4 , wherein the multiprocessor is to concurrently execute a first floating-point operation at a first precision on the first compute unit and a second floating-point operation at a second precision.
6. The graphics processing unit as in claim 1 , the first compute unit additionally including one or more shifters to normalize or align an intermediate result.
7. The graphics processing unit as in claim 6 , the first compute unit additionally configurable to compute a 16-bit sum based on the intermediate product.
8. A data processing system comprising:
a graphics processing unit to accelerate machine-learning operations, the graphics processing unit including a multiprocessor having a single instruction, multiple thread (SIMT) architecture, the multiprocessor to execute at least one single instruction across multiple threads of the multiprocessor; and
a first compute unit included within the multiprocessor, the at least one single instruction to cause the first compute unit to perform a two-dimensional matrix multiply and accumulate operation, wherein to perform the two-dimensional matrix multiply and accumulate operation includes to compute an intermediate product of 16-bit operands and to compute a 32-bit sum based on the intermediate product; and
a memory communicatively coupled with the graphics processing unit;
wherein to compute a 32-bit sum based on the intermediate product, the first compute unit is to:
perform a floating-point multiply of two or more 16-bit operands at a first intermediate precision to generate the intermediate product, the first intermediate precision less than 32 bits;
compute a sum based on the intermediate product to generate an intermediate sum at a second intermediate precision; and
compute the 32-bit sum via a conversion of the intermediate sum at the second intermediate precision to a 32-bit precision.
9. The data processing system as in claim 8 , the multiprocessor to execute parallel threads of a thread group, each thread of the thread group having independent thread state.
10. The data processing system as in claim 9 , the multiprocessor including a scheduler to schedule the parallel threads to multiple compute units within the multiprocessor.
11. The graphics processing unit as in claim 10 , the multiple compute units within the multiprocessor including a second compute unit to perform an integer operation, the scheduler to schedule a floating-point operation to the first compute unit and an integer operation to the second compute unit, wherein the multiprocessor is to concurrently execute a floating-point operation on the first compute unit and an integer operation on the second compute unit.
12. The graphics processing unit as in claim 11 , the multiprocessor to concurrently execute, on the first compute unit a first floating-point operation at a first precision and a second floating point operation at a second precision.
13. The graphics processing unit as in claim 8 , the first compute unit additionally including one or more shifters to normalize or align an intermediate result.
14. The graphics processing unit as in claim 13 , the first compute unit to additionally configurable compute a 16-bit sum based on the intermediate product.
15. A method of accelerating a machine-learning operation, the method comprising:
decoding a single instruction on a graphics processing unit (GPU), the GPU having a single instruction, multiple thread (SIMT) architecture;
executing the single instruction via a multiprocessor within the GPU, the single instruction executed across multiple threads of the multiprocessor; and
in response to executing the single instruction via the multiprocessor, performing a two-dimensional matrix multiply and accumulate operation on a first compute unit of the multiprocessor, wherein performing the two-dimensional matrix multiply and accumulate operation includes computing an intermediate product of 16-bit operands and computing a 32-bit sum based on the intermediate product, wherein computing the intermediate product includes:
performing a floating-point multiply of two or more 16-bit operands at a first intermediate precision to generate the intermediate product, the first intermediate precision less than 32-bits;
compute a sum based on the intermediate product to generate an intermediate sum at a second intermediate precision; and
compute the 32-bit sum via a conversion of the intermediate sum at the second intermediate precision to a 32-bit precision.
16. The method as in claim 15 , additionally comprising executing parallel threads of a thread group, each thread of the thread group having independent thread state.
17. The method as in claim 16 , additionally comprising scheduling the parallel threads of the thread group to multiple compute units within the multiprocessor.
18. The method as in claim 17 , additionally comprising:
scheduling a floating-point operation to the first compute unit and an integer operation to a second compute unit; and
performing the integer operation via a second compute unit within the multiprocessor concurrently with the floating-point operation on the first compute unit.
19. The method as in claim 18 , additionally comprising:
concurrently executing, on the first compute unit, a first floating-point operation at a first precision and a second floating-point operation at a second precision.
20. The method as in claim 15 , additionally comprising:
computing a 16-bit sum based on the intermediate product.