Machine learning sparse computation mechanism
Techniques to improve performance of matrix multiply operations are described in which a compute kernel can specify one or more element-wise operations to perform on output of the compute kernel before the output is transferred to higher levels of a processor memory hierarchy.
1. An apparatus comprising:
a system interconnect;
a graphics processor coupled with the system interconnect, the graphics processor including:
first circuitry to load elements of matrices into a first memory of the graphics processor, wherein the first memory is a global memory;
second circuitry to transfer a subset of elements of the matrices from the first memory into a second memory, the second memory local to a set of processing resources of the graphics processor;
third circuitry configurable to execute a compute kernel on the graphics processor, the compute kernel to perform a matrix multiply operation on the subset of elements of the matrices, the compute kernel to specify one or more element-wise operations to perform on output of the matrix multiply operation before the output is transferred to the first memory, and the one or more element-wise operations include to apply an activation function to the output; and
fourth circuitry to perform the one or more element-wise operations on the output of the matrix multiply operation while the output is stored in the second memory.
2. The apparatus as in claim 1 , wherein the third circuitry is configured to perform the matrix multiply operation in response to a first instruction.
3. The apparatus as in claim 2 , wherein the matrix multiply operation is a sparse matrix multiply operation.
4. The apparatus as in claim 2 , wherein the matrix multiply operation is associated with a convolution operation to be performed via the graphics processor.
5. The apparatus as in claim 2 , wherein the first instruction is to specify to the fourth circuitry the one or more element-wise operations to perform on the output of the matrix multiply operation.
6. The apparatus as in claim 2 , wherein the fourth circuitry is configured execute a second instruction, the second instruction to specify one or more element-wise operations to perform on the output of the matrix multiply operation.
7. The apparatus as in claim 1 , wherein the one or more element-wise operations additionally include to perform a bias operation before performing the activation function.
8. The apparatus as in claim 7 , wherein the activation function is a rectified linear unit function.
9. The apparatus as in claim 1 , comprising fifth circuitry to performing a pooling operation after performance of the activation function.
10. A non-transitory machine-readable medium including instructions,
execution of which by one or more processors, causes the one or more processors to perform operations comprising:
loading elements of matrices into a first memory of a graphics processor of the one or more processors, wherein the first memory is a global memory of the graphics processor;
transferring a subset of elements of the matrices from the first memory into a second memory of the graphics processor, the second memory local to a set of processing resources of the graphics processor; and
triggering execution of a compute kernel on the graphics processor, wherein the compute kernel performs a matrix multiply operation on the subset of elements of the matrices, and a matrix multiply framework enables the compute kernel to specify one or more element-wise operations to perform on output of the matrix multiply operation before the output is transferred to the first memory, the one or more element-wise operations including applying an activation function to the output and the matrix multiply framework is to provide a primitive to enable the compute kernel to specify the one or more element-wise operations to perform on the output of the matrix multiply operation.
11. The non-transitory machine-readable medium as in claim 10 , wherein the one or more element-wise operations are performed after the matrix multiply operation is complete and while the output is stored in the second memory.
12. The non-transitory machine-readable medium as in claim 10 , wherein the operations additionally include performing a pooling operation after performing the activation function.
13. The non-transitory machine-readable medium as in claim 12 , wherein the activation function is a rectified linear unit function.
14. A graphics processing system comprising:
a memory device;
a graphics processor coupled with the memory device, the graphics processor including:
first circuitry to load elements of matrices into a first memory of the graphics processor, wherein the first memory is a global memory;
second circuitry to transfer a subset of elements of the matrices from the first memory into a second memory, the second memory local to a set of processing resources of the graphics processor;
third circuitry configurable to execute a compute kernel on the graphics processor, the compute kernel to perform a matrix multiply operation on the subset of elements of the matrices, the compute kernel to specify one or more element-wise operations to perform on output of the matrix multiply operation before the output is transferred to the first memory, and the one or more element-wise operations include to apply an activation function to the output; and
fourth circuitry to perform the one or more element-wise operations on the output of the matrix multiply operation while the output is stored in the second memory.
15. The graphics processing system as in claim 14 , wherein the third circuitry is configured to perform the matrix multiply operation in response to a first instruction and the matrix multiply operation is a sparse matrix multiply operation.
16. The graphics processing system as in claim 15 , wherein the matrix multiply operation is associated with a convolution operation to be performed via the graphics processor.
17. The graphics processing system as in claim 15 , wherein the first instruction is to specify to the fourth circuitry the one or more element-wise operations to perform on the output of the matrix multiply operation.
18. The graphics processing system as in claim 15 , wherein the fourth circuitry is configured execute a second instruction, the second instruction to specify one or more element-wise operations to perform on the output of the matrix multiply operation.
19. The graphics processing system as in claim 14 , wherein the one or more element-wise operations additionally include to perform a bias operation before performance of the activation function and the activation function is a rectified linear unit function.
20. The graphics processing system as in claim 14 , comprising fifth circuitry to performing a pooling operation after performance of the activation function.