Techniques for near data acceleration for a multi-core architecture
Examples include techniques for near data acceleration for a multi-core architecture. A near data processor included in a memory controller of a processor may access data maintained in a memory device coupled with the near data processor via one or more memory channels responsive to a work request to execute a kernel, an application or a loop routine using the accessed data to generate values. The near data processor provides an indication to the requestor of the work request that values have been generated.
1. An apparatus comprising:
a memory controller at a processor, the memory controller to be configured to couple with multiple cores of the processor via an on-die interconnect, the memory controller to control access to a first memory device, the memory controller to include a near data processor, the near data processor to include:
circuitry to execute a kernel, an application, or a loop routine using data maintained in the first memory device, the data directly accessible to the near data processor via at least one memory channel coupled with the first memory device; and
a plurality of memory buffers arranged to receive a work request from a core from among the multiple cores and arranged to indicate that values have been generated by the circuitry responsive to the work request.
2. The apparatus of claim 1 , further comprising the memory controller to be configured to couple with a second memory controller at the processor via the on-die interconnect, the second memory controller to control access to a second memory device, wherein the near data processor is arranged to serve as an agent on the on-die interconnect to access data maintained in the second memory device through the second memory controller, the circuitry to execute at least a portion of the kernel, the application, or the loop routine using the accessed data maintained in the second memory device.
3. The apparatus of claim 1 , the data maintained in the first memory device comprises the data maintained in a memory region of the first memory device shared with the core from among the multiple cores, wherein the circuitry is further to:
send a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core;
concurrently send an access request to the first memory device to access the data maintained in the memory region;
receive, responsive to the coherency request, an indication that the data is not included in the cache for the core;
receive, responsive to the access request, the data maintained in the memory region; and
use the data maintained in the memory region to execute the kernel, the application, or the loop routine.
4. The apparatus of claim 1 , the data maintained in the first memory device comprises the data maintained in a memory region of the first memory device shared with the core from among the multiple cores, wherein the circuitry is further to:
send a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core;
concurrently send an access request to the first memory device to access the data maintained in the memory region;
receive, responsive to the coherency request, an indication that the data is included in the cache for the core; and
use the data included in the cache for the core to execute the kernel, the application, or the loop routine.
5. The apparatus of claim 1 , comprising the generated values to be stored to a memory region of the first memory device shared with the core from among the multiple cores, wherein the circuitry is further to:
send a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region;
receive, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes data from the memory region;
send an indication to the coherency agent that data included in the cache line is invalid; and
cause the values to be stored to the memory region.
6. The apparatus of claim 1 , comprising the generated values to be stored to a memory region of the first memory device shared with the core from among the multiple cores, wherein the circuitry is further to:
send a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region where the generated values are to be stored;
concurrently send an access request to the first memory device to access the data maintained in the memory region that corresponds to the cache line;
receive, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes only a portion of the data from the memory region;
receive, responsive to the access request, the data maintained in the memory region;
send an indication to the coherency agent that the cache line includes invalid data; and
cause the values to be stored to the memory region.
7. The apparatus of claim 1 , the first memory device comprising a high bandwidth memory (HBM) stack resident on a separate chip from the memory controller and the multiple cores of the processor, wherein the HBM stack includes dynamic random access memory.
8. The apparatus of claim 7 , comprising the data directly accessible to the near data processor via the at least one memory channel coupled with the HBM stack, wherein the at least one memory channel is arranged to operate in compliance with a Joint Electronic Device Engineering Council (JEDEC) specification to include HBM version 2, JESD235C.
9. A method comprising:
receiving, at a near data processor included in a memory controller at a processor, a work request to execute a kernel, an application, or a loop routine using data maintained in a memory region of a first memory device coupled with the near data processor via at least one memory channel;
accessing the data maintained in the memory region to generate values responsive to execution of the kernel, the application, or the loop routine; and
indicating to a requester of the work request that the values have been generated responsive to the work request.
10. The method of claim 9 , comprising receiving the work request in a memory buffer of the near data processor, wherein circuitry of the near data processor is arranged to execute the kernel, the application or the loop routine using the data maintained in the memory region of the first memory device.
11. The method of claim 10 , the requester of the work request comprising a core of the processor, the core coupled with the memory controller via an on-die interconnect.
12. The method of claim 11 , comprising the memory controller at the processor to couple with a second memory controller at the processor via the on-die interconnect, the second memory controller to control access to a second memory device, wherein the near data processor is arranged to serve as an agent on the on-die interconnect to access data maintained in the second memory device through the second memory controller, the circuitry to execute at least a portion of the kernel, the application, or the loop routine using the accessed data maintained in the second memory device.
13. The method of claim 11 , the data maintained in the first memory device comprises the data maintained in a memory region of the first memory device shared with the core of the processor, the method further comprising the near data processor:
sending a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core;
concurrently sending an access request to the first memory device to access the data maintained in the memory region;
receiving, responsive to the coherency request, an indication that the data is not included in the cache for the core;
receiving, responsive to the access request, the data maintained in the memory region; and
using the data maintained in the memory region to execute the kernel, the application, or the loop routine.
14. The method of claim 11 , the data maintained in the first memory device comprises the data maintained in a memory region of the first memory device shared with the core of the processor, the method further comprising the near data processor:
sending a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core;
concurrently sending an access request to the first memory device to access the data maintained in the memory region;
receiving, responsive to the coherency request, an indication that the data is included in the cache for the core; and
using the data included in the cache for the core to execute the kernel, the application, or the loop routine.
15. The method of claim 11 , comprising the values to be stored to a memory region of the first memory device shared with the core of the processor, the method further comprising the near data processor:
sending a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region;
receiving, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes data from the memory region;
sending an indication to the coherency agent that data included in the cache line is invalid; and
causing the values to be stored to the memory region.
16. The method of claim 11 , comprising the values to be stored to a memory region of the first memory device shared with the core of the processor, the method further comprising the near data processor:
sending a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region where the values are to be stored;
concurrently sending an access request to the first memory device to access the data maintained in the memory region that corresponds to the cache line;
receiving, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes only a portion of the data from the memory region;
receiving, responsive to the access request, the data maintained in the memory region;
sending an indication to the coherency agent that the cache line includes invalid data; and
causing the values to be stored to the memory region.
17. The method of claim 11 , the first memory device comprising a high bandwidth memory (HBM) stack resident on a separate chip from the memory controller and the core of the processor, wherein the HBM stack includes dynamic random access memory.
18. A system comprising:
a first memory device;
a plurality of cores of a processor; and
a first memory controller at the processor, the first memory controller to be configured to couple with the plurality of cores via an on-die interconnect, the first memory controller to control access to the first memory device, the first memory controller to include a near data processor, the near data processor to include:
circuitry to execute a kernel, an application, or a loop routine using data maintained in the first memory device, the data directly accessible to the near data processor via at least one memory channel coupled with the first memory device; and
a plurality of memory buffers arranged to receive a work request from a core from among the plurality of cores and arranged to indicate that values have been generated by the circuitry responsive to the work request.
19. The system of claim 18 , further comprising:
a second memory device; and
a second memory controller at the processor, the first memory controller to be configured to couple with the plurality of cores and the first memory controller via the on-die interconnect, the second memory controller to control access to the second memory device, wherein the near data processor is arranged to serve as an agent on the on-die interconnect to access data maintained in the second memory device through the second memory controller, the circuitry to execute at least a portion of the kernel, the application, or the loop routine using the accessed data maintained in the second memory device.
20. The system of claim 18 , the data maintained in the first memory device comprises the data maintained in a memory region of the first memory device shared with the core from the plurality of cores, wherein the circuitry is further to:
send a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core;
concurrently send an access request to the first memory device to access the data maintained in the memory region;
receive, responsive to the coherency request, an indication that the data is not included in the cache for the core;
receive, responsive to the access request, the data maintained in the memory region; and
use the data maintained in the memory region to execute the kernel, the application, or the loop routine.
21. The system of claim 18 , comprising the generated values to be stored to a memory region of the first memory device shared with the core from the plurality of cores, wherein the circuitry is further to:
send a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region;
receive, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes data from the memory region;
send an indication to the coherency agent that data included in the cache line is invalid; and
cause the values to be stored to the memory region.
22. The system of claim 18 , comprising the generated values to be stored to a memory region of the first memory device shared with the core from the plurality of cores, wherein the circuitry is further to:
send a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region where the generated values are to be stored;
concurrently send an access request to the first memory device to access the data maintained in the memory region that corresponds to the cache line;
receive, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes only a portion of the data from the memory region;
receive, responsive to the access request, the data maintained in the memory region;
send an indication to the coherency agent that the cache line includes invalid data; and
cause the values to be stored to the memory region.
23. The system of claim 22 , the first memory device comprising a high bandwidth memory (HBM) stack resident on a separate chip from the first memory controller and the plurality of cores of the processor, wherein the HBM stack includes dynamic random access memory.