Local memory sharing between kernels
One embodiment provides for a general-purpose graphics processing unit comprising a set of processing elements to execute one or more thread groups of a second kernel to be executed by the general-purpose graphics processor, an on-chip memory coupled to the set of processing elements, and a scheduler coupled with the set of processing elements, the scheduler to schedule the thread groups of the kernel to the set of processing elements, wherein the scheduler is to schedule a thread group of the second kernel to execute subsequent to a thread group of a first kernel, the thread group of the second kernel configured to access a region of the on-chip memory that contains data written by the thread group of the first kernel in response to a determination that the second kernel is dependent upon the first kernel.
1. A general-purpose graphics processor comprising:
a set of processing elements to execute one or more thread groups of a second kernel to be executed by the general-purpose graphics processor;
an on-chip memory coupled to the set of processing elements; and
a scheduler coupled with the set of processing elements, the scheduler to:
schedule the thread groups of the second kernel to the set of processing elements, wherein the scheduler is to schedule a second thread group of the second kernel to execute subsequent to a first thread group of a first kernel and, in response to a determination that the second kernel is dependent upon the first kernel and that the first thread group and the second thread group have a same number of threads, the second thread group is configured to access a region of the on-chip memory that contains data written by the first thread group, wherein the scheduler is to clear at least a portion of the on-chip memory before execution of a third thread group of a third kernel in response to a determination that the third thread group has a different number of threads than the first thread group and the second thread group.
2. The general-purpose graphics processor as in claim 1 , wherein the scheduler is to clear at least a portion of the on-chip memory before execution of the third thread group in response to a determination that the third kernel is not dependent upon the first kernel or the second kernel.
3. The general-purpose graphics processor as in claim 1 , wherein the scheduler is to bypass a clear of the region of the on-chip memory that contains data written by the first thread group in response to the determination that the second kernel is dependent upon the first kernel.
4. The general-purpose graphics processor as in claim 3 , wherein the first kernel is to compute output of a first layer of a neural network and write output data to the on-chip memory.
5. The general-purpose graphics processor as in claim 4 , wherein the second kernel is to read the output data from the on-chip memory and compute output of a second layer of a neural network, the first layer of the neural network connected to the second layer of the neural network.
6. The general-purpose graphics processor as in claim 1 , wherein the on-chip memory includes an implicitly managed cache memory and an explicitly managed shared memory.
7. A circuit board comprising:
a host interconnect;
a general-purpose graphics processor coupled to the host interconnect, the general-purpose graphics processor including a set of processing elements to execute one or more thread groups of a second kernel to be executed by the general-purpose graphics processor, an on-chip memory coupled to the set of processing elements, and a scheduler coupled with the set of processing elements, the scheduler to schedule the thread groups of the second kernel to the set of processing elements, wherein the scheduler is to schedule a second thread group of the second kernel to execute subsequent to a first thread group of a first kernel and, in response to a determination that the second kernel is dependent upon the first kernel and that the first thread group and the second thread group have a same number of threads, the second thread group is configured to access a region of the on-chip memory that contains data written by the first thread group, wherein the scheduler is to clear at least a portion of the on-chip memory before execution of a third thread group of a third kernel in response to a determination that the third thread group has a different number of threads than the first thread group and the second thread group; and
a memory coupled to the host interconnect and the general-purpose graphics processor.
8. The circuit board as in claim 7 , wherein the scheduler is to clear at least a portion of the on-chip memory before execution of a third thread group of a third kernel in response to a determination that the third kernel is not dependent upon the first kernel or the second kernel.
9. The circuit board as in claim 8 , wherein the scheduler is to bypass a clear of the region of the on-chip memory that contains data written by the first thread group in response to the determination that the second kernel is dependent upon the first kernel.
10. The circuit board as in claim 9 , wherein the first kernel is to compute output of a first layer of a neural network and write output data to the on-chip memory, and wherein the second kernel is to read the output data from the on-chip memory and compute output of a second layer of a neural network, the first layer of the neural network connected to the second layer of the neural network.
11. The circuit board as in claim 7 , wherein the on-chip memory includes an implicitly managed cache memory and an explicitly managed shared memory.
12. A method on a parallel processor, the method comprising:
scheduling thread groups of a second kernel to a set of processing elements via a scheduler of the parallel processor, wherein scheduling thread groups of the second kernel includes scheduling a second thread group of the second kernel to execute subsequent to a first thread group of a first kernel; and
in response to determining that the second kernel is dependent upon the first kernel and that the first thread group and the second thread group have a same number of threads, configuring the second thread group to access a region of the on-chip memory that contains data written by the first thread group; and
clearing at least a portion of the on-chip memory before executing a third thread group of a third kernel in response to a determination that the third thread group has a different number of threads than the first thread group and the second thread group.
13. The method as in claim 12 , further comprising clearing at least a portion of the on-chip memory before execution of the third thread group in response to a determination that the third kernel is not dependent upon the first kernel or the second kernel.
14. The method as in claim 12 , further comprising bypassing a clear of the region of the on-chip memory that contains data written by the first thread group in response to the determination that the second kernel is dependent upon the first kernel.
15. The method as in claim 14 , further comprising computing, via the first kernel, output of a first layer of a neural network and write output data to the on-chip memory.
16. The method as in claim 15 , further comprising reading, via the second kernel, the output data from the on-chip memory and computing output of a second layer of a neural network, the first layer of the neural network connected to the second layer of the neural network.
17. The method as in claim 16 , wherein the on-chip memory includes an implicitly managed cache memory and an explicitly managed shared memory.