System, apparatus and method for processing remote direct memory access operations with a device-attached memory
In one embodiment, a processor includes: one or more cores to execute instructions; at least one cache memory; and a coherence circuit coupled to the at least one cache memory. The coherence circuit may have a direct memory access circuit to receive a write request, and based at least in part on an address of the write request, to directly send the write request to a device coupled to the processor via a first bus, to cause the device to store data of the write request to a device-attached memory. Other embodiments are described and claimed.
1. A processor comprising:
one or more cores to execute instructions;
at least one cache memory; and
a coherence circuit coupled to the at least one cache memory, the coherence circuit having a direct memory access (DMA) circuit to receive a write request from a network interface card (NIC), and based at least in part on an address of the write request, to directly send the write request to a device coupled to the processor via a first bus and not to a system memory coupled to the processor, to cause the device to store data of the write request to a device-attached memory locally coupled to the device, wherein when the write request has a set relaxed ordering indicator, the DMA circuit is to directly send the write request to the device without a prefetch operation by the processor to obtain ownership.
2. The processor of claim 1 , wherein the coherence circuit comprises an address range decoder, and the DMA circuit is to access the address range decoder using the address of the write request to determine that the write request is directed to the device-attached memory.
3. The processor of claim 1 , wherein the write request comprises a remote direct memory access request having an unset transaction layer processing hint.
4. The processor of claim 1 , wherein in response to a determination that the address of the write request is directed to the system memory coupled to the processor, the DMA circuit is to store the data of the write request in the at least one cache memory.
5. The processor of claim 4 , wherein the DMA circuit is to store the data of the write request in the at least one cache memory further in response to an allocating platform default setting.
6. The processor of claim 1 , wherein the coherence circuit, in response to a use once indicator from the device for a second data, is to cause the processor to use the second data one time, and thereafter send a second write request comprising the second data to the device.
7. The processor of claim 6 , wherein in response to a completion from the device for the second write request, the processor is to send a forward message to the device to cause the device to change a bias of the second data to a device bias.
8. The processor of claim 1 , wherein the coherence circuit, in response to a third write request having an unset relaxed ordering indicator, is to order the third write request after one or more prior write requests.
9. A non-transitory machine-readable medium having stored thereon instructions, which if performed by a machine cause the machine to perform a method comprising:
receiving, in a control circuit of a device coupled to a host processor via a first bus, an invalidation request for a first data block from the host processor;
in response to the invalidation request, updating a bias for the first data block from a device bias to a temporary host bias, to prevent a device-initiated request to the first data block; and
thereafter sending a completion message to the host processor comprising a use once indicator to cause the host processor to use the first data block one time and then to relinquish ownership of the first data block to the device.
10. The non-transitory machine-readable medium of claim 9 , wherein the method further comprises:
receiving, in the device, a write request for the first data block;
in response to the write request, storing the first data block in a device-attached memory; and
sending a second completion message to the host processor.
11. The non-transitory machine-readable medium of claim 10 , wherein after sending the second completion message to the host processor, receiving a forward message from the host processor and in response to the forward message, updating the bias for the first data block to the device bias.
12. The non-transitory machine-readable medium of claim 11 , wherein the method further comprises updating the bias for the first data block to the device bias without issuing a request for ownership to the host processor.
13. The non-transitory machine-readable medium of claim 9 , wherein the method further comprises:
receiving, in the control circuit of the device, a second write request for a second data block, the second write request comprising a direct write request; and
in response to the second write request, storing the second data block in the device-attached memory.
14. The non-transitory machine-readable medium of claim 9 , wherein the first data block comprises remote direct memory access data from a first input/output device coupled to the device via the host processor and a network interface controller.
15. A system comprising:
a host processor having at least one core, at least one cache memory and a coherence circuit coupled to the at least one cache memory, wherein the coherence circuit comprises a direct memory access (DMA) circuit to receive a remote direct memory access (RDMA) write request, and based at least in part on an address of the RDMA write request, to directly send the RDMA write request to a device, wherein the RDMA write request has a relaxed ordering indicator associated therewith;
a host memory coupled to the host processor;
the device coupled to the host processor via a first bus, wherein in response to the RDMA write request, the device is to store data of the RDMA write request to a device-attached memory and to set a bias of the data to a device bias; and
the device-attached memory locally coupled to the device.
16. The system of claim 15 , wherein the coherence circuit comprises an address range decoder, and the DMA circuit is to access the address range decoder using the address of the RDMA write request to determine that the write request is directed to the device-attached memory, wherein the system comprises a server platform having an allocating default setting.
17. The system of claim 16 , wherein, in response to a second write request directed to the host memory coupled to the host processor, the host processor is to store second data of the second write request to the least one cache memory based on the allocating default setting.
18. The system of claim 15 , wherein, in response to a second RDMA write request directed to the device-attached memory, the host processor is to send an invalidation message to the device, and in response to the invalidation message, the device is to send a use once response to the host processor to grant a temporary host bias for second data.
19. The system of claim 18 , wherein the host processor, in response to the use once response, is to use the second data of the second RDMA write request one time, and thereafter send a write request to the device to cause the device to store the second data in the device-attached memory, and thereafter the host processor is to perform a self-invalidation of the second data to relinquish the temporary host bias.