Compute optimization mechanism for deep neural networks
An apparatus to facilitate compute optimization is disclosed. The apparatus includes a memory device including a first integrated circuit (IC) including a plurality of memory channels and a second IC including a plurality of processing units, each coupled to a memory channel in the plurality of memory channels.
1. An apparatus, comprising:
a graphics processor; and
a memory device coupled with the graphics processor, the memory device including:
a first integrated circuit (IC) including:
a plurality of memory channels,
a plurality of high-speed interfaces, each of the plurality of memory channels being coupled to a respective interface of the plurality of high-speed interfaces, and
a memory;
a second IC including a plurality of processing units, each processing unit of the plurality of processing units being coupled to a respective memory channel of the plurality of memory channels via the respective interface for the memory channel; and
a controller, the controller to facilitate access by the plurality of processing units to the plurality of memory channels to perform compute operations in response to an instruction from the graphics processor, the compute operations to be performed without transfer of data from the memory to the graphics processor.
2. The apparatus of claim 1 , wherein the compute operations include data intensive functions.
3. The apparatus of claim 2 , wherein the compute operations include operations for a deep neural network.
4. The apparatus of claim 1 , wherein the memory device includes:
a first memory channel of the plurality of memory channels;
a first interface of the plurality of high-speed interfaces, the first interface being coupled to the first memory channel; and
a first processing unit of the plurality of processing units coupled to the first interface.
5. The apparatus of claim 4 , wherein the controller is to facilitate access of the first memory channel by the first processing unit via the first interface to perform the compute operations.
6. The apparatus of claim 5 , wherein the controller is further to facilitate storing of results of the compute operations to the memory via the first memory channel by the first processing unit.
7. The apparatus of claim 6 , wherein the controller is further to facilitate access by the graphics processor to the results of the compute operations stored in the memory via the first memory channel.
8. The apparatus of claim 1 , wherein the memory comprises a high bandwidth memory (HBM), each of the plurality of memory channels being coupled to the HBM.
9. An integrated circuit (IC) package, comprising:
a first integrated circuit (IC) coupled with a graphics processor, the first IC including:
a plurality of memory channels,
a plurality of high-speed interfaces, each of the plurality of high-speed interfaces being coupled to a respective memory channel of the plurality of memory channels, and
a memory;
a second IC including a plurality of processing units, each processing unit of the plurality of processing units being coupled to a memory channel of the plurality of memory channels via the respective interface for the memory channel, and
a controller, the controller to facilitate access by the plurality of processing units to the plurality of memory channels to perform compute operations in response to an instruction from the graphics processor, the compute operations to be performed without transfer of data from the memory to the graphics processor.
10. The IC package of claim 9 , wherein the compute operations include data intensive functions.
11. The IC package of claim 10 , wherein the compute operations include operations for a deep neural network.
12. The IC package of claim 9 , wherein the controller is further to facilitate storing of results of the compute operations to the memory via each of the plurality of memory channels by the respective processing units of the plurality of processing units.
13. The IC package of claim 12 , wherein the controller is further to facilitate access by the graphics processor to the results of the compute operations stored in the memory via the respective memory channels of the plurality of memory channels.
14. The IC package of claim 9 , wherein the memory comprises a high bandwidth memory (HBM), each of the plurality of memory channels being coupled to the HBM.
15. A system, comprising:
a graphics processing unit (GPU); and
a memory device coupled with the GPU, the memory device including:
a memory integrated circuit (IC) comprising:
a plurality of memory channels,
a plurality of high speed interfaces, each of the plurality of memory channels being coupled to a respective interface of the plurality of high speed interfaces, and
a high bandwidth memory (HBM), each of the plurality of memory channels being coupled to the HBM;
a processor IC including a plurality of processing units, each processing unit of the plurality of processing units being coupled to a respective memory channel of the plurality of memory channels via the respective interface for the memory channel; and
a controller, the controller to facilitate access by the plurality of processing units to the plurality of memory channels to perform compute operations in response to an instruction from the GPU, the compute operations to be performed without transfer of data from the memory to the GPU.
16. The system of claim 15 , wherein the compute operations include data intensive functions.
17. The system of claim 16 , wherein the compute operations include operations for a deep neural network.
18. The system of claim 15 , further comprising:
a first memory channel of the plurality of memory channels;
a first interface of the plurality of high speed interfaces, the first interface being coupled to the first memory channel; and
a first processing unit of the plurality of processing units coupled to the first interface.
19. The system of claim 18 , wherein the memory device further comprises a controller is to facilitate access of the first memory channel by the first processing unit via the first interface to perform the compute operations.
20. The system of claim 19 , wherein the controller is further to facilitate storing of results of the compute operations to the HBM via the first memory channel by the first processing unit.
21. The system of claim 20 , wherein the controller is further to facilitate access by the GPU to the compute operations results stored in the HBM via the first memory channel.