TECHNIQUES TO SHARE MEMORY ACROSS NODES IN A SYSTEM
Techniques to shared system memory across nodes in a system. Circuitry is arranged to provide a mechanism to share a memory region of a memory maintained at a first host CPU at a first node across multiple other host CPUs at multiple other nodes using various links and protocols described in one or more revisions of the Compute Express Link (CXL) specification.
1 . An apparatus comprising:
an interface arranged to couple with a plurality of communication links that are arranged to separately couple with a first host central processing unit (CPU) at a first node, a second host CPU at a second node and a third host CPU at a third node, wherein the first host CPU is to be arranged to share a region of memory maintained at the first node with the second host CPU and the third host CPU, the shared region of memory to be mapped by the second and third CPUs as host-managed device memory (HDM) device coherent using back-invalidation (HDM-DB); and
circuitry configured to:
receive, through the interface from the first host CPU, a read or write request to a cache line mapped to the shared region of memory via a host to device (H2D) message;
determine that the second host CPU has a latest version of the cache line and use a back-invalidate message exchange with the second host CPU to resolve coherency; and
send, through the interface to the first host CPU, a device to host (D2H) message to include the latest version of the cache line and an indication that the second host CPU does not have a cached copy of the cache line.
2 . The apparatus of claim 1 , wherein the circuitry configured to use the back-invalidate message exchange comprises the circuitry to send, through the interface, subordinate to master (S2M) messages to the second host CPU and receive, through the interface, master to subordinate (M2S) messages from the second host CPU to resolve coherency.
3 . The apparatus of claim 1 , wherein the interface and the circuitry are included in an extended Compute Express Link (CXL) bridge (ECB) and the plurality of communication links are arranged to operate using CXL specification protocols.
4 . The apparatus of claim 3 , wherein the ECB is located at a CXL switch and is arranged to be implemented by CXL switch circuitry.
5 . The apparatus of claim 3 , wherein the ECB is included in a die located at a CXL switch.
6 . The apparatus of claim 3 , wherein the shared region of memory to be mapped by the second and third CPUs as HDM-DB comprises the shared region of memory to be CXL.mem based HDM-DB mapped shared memory.
7 . At least one non-transitory computer-readable storage medium, comprising a plurality of instructions, that when executed, cause circuitry to:
couple, through an interface, with a plurality of communication links that are arranged to separately couple with a first host central processing unit (CPU) at a first node, a second host CPU at a second node and a third host CPU at a third node, wherein the first host CPU is to be arranged to share a region of memory maintained at the first node with the second host CPU and the third host CPU, the shared region of memory to be mapped by the second and third CPUs as host-managed device memory (HDM) device coherent using back-invalidation (HDM-DB);
receive, through the interface from the first host CPU, a read or write request to a cache line mapped to the shared region of memory via a host to device (H2D) message;
determine that the second host CPU has a latest version of the cache line and use a back-invalidate message exchange with the second host CPU to resolve coherency; and
send, through the interface to the first host CPU, a device to host (D2H) message to include the latest version of the cache line and an indication that the second host CPU does not have a cached copy of the cache line.
8 . The least one non-transitory computer-readable storage medium of claim 7 , wherein to use the back-invalidate message exchange comprises to send, through the interface, subordinate to master (S2M) messages to the second host CPU and receive, through the interface, master to subordinate (M2S) messages from the second host CPU to resolve coherency.
9 . The least one non-transitory computer-readable storage medium of claim 7 , wherein the interface is arranged to operate using Compute Express Link (CXL) specification protocols.
10 . The least one non-transitory computer-readable storage medium of claim 9 , wherein the shared region of memory to be mapped by the second and third CPUs as HDM-DB comprises the shared region of memory to be CXL.mem based HDM-DB mapped shared memory.
11 . An apparatus of comprising:
an interface arranged to couple with a plurality of communication links that are arranged to separately couple with a first host central processing unit (CPU) at a first node, a second host CPU at a second node and a third host CPU at a third node, wherein the first host CPU is to be arranged to share a region of memory maintained at the first node with the second host CPU and the third host CPU, the shared region of memory to be mapped by the second and third CPUs as host-managed device memory (HDM) device coherent using back-invalidation (HDM-DB); and
circuitry configured to:
receive, from the second host CPU, a read or write request to a cache line mapped to the shared region of memory via a master to subordinate (M2S) message;
determine that the third host CPU has a latest version of the cache line and use a back-invalidate message exchange with the third host CPU to resolve coherency;
translate the M2S message to a device to host (D2H) message and forward the D2H message to the first host CPU to read or write data to the cache line;
receive the cache line data from first host CPU via a host to device (H2D) message; and
translate the H2D message to a subordinate to master (S2M) message and forward the S2M message to provide the cache line data that was written to or read from the cache line to the second host CPU.
12 . The apparatus of claim 11 , wherein the interface and the circuitry are included in an extended Compute Express Link (CXL) bridge (ECB) and the plurality of communication links are arranged to operate using CXL specification protocols.
13 . The apparatus of claim 12 , wherein the ECB is located at a CXL switch and is arranged to be implemented by CXL switch circuitry.
14 . The apparatus of claim 12 , wherein the ECB is included in a die located at a CXL switch.
15 . The apparatus of claim 12 , wherein the shared region of memory to be mapped by the second and third CPUs as HDM-DB comprises the shared region of memory to be CXL.mem based HDM-DB mapped shared memory.
16 . An apparatus comprising:
an interface arranged to couple with a plurality of communication links that are arranged to separately couple with a first host central processing unit (CPU) at a first node and a second host CPU at a second node, wherein the first host CPU is to be arranged to share a region of memory maintained at the first node with the second host CPU, the shared region of memory to be mapped by the second CPU as host-managed device memory (HDM) device coherent (HDM-D); and
circuitry configured to:
receive, through the interface from the first host CPU, a read or write request to a cache line mapped to the shared region of memory via a host to device (H2D) message;
determine that the second host CPU has a latest version of the cache line; and
send, through the interface to the first host CPU, a device to host (D2H) message to include the latest version of the cache line and an indication that the second host CPU does not have a cached copy of the cache line.
17 . The apparatus of claim 16 , wherein to determine that the second host CPU has the latest version of the cache line includes the circuitry configured to:
translate the H2D message received from the first host CPU to a second D2H message, the second D2H message to indicate an address for the cache line in the shared region of memory for which the first host CPU has requested to read or write;
send, through the interface, the second D2H message to the second host CPU; and
receive, through the interface from the second host CPU, a response to indicate that the second host CPU does not have a cached copy of the cache line via a second H2D message.
18 . The apparatus of claim 16 , wherein the interface and the circuitry are included in an extended Compute Express Link (CXL) bridge (ECB) and the plurality of communication links are arranged to operate using CXL specification protocols.
19 . The apparatus of claim 18 , wherein the ECB is located at a CXL switch and is arranged to be implemented by CXL switch circuitry.
20 . The apparatus of claim 18 , wherein the ECB is included in a die located at a CXL switch.
21 . The apparatus of claim 18 , wherein the shared region of memory to be mapped by the second CPU as HDM-D comprises the shared region of memory to be CXL.mem based HDM-D mapped shared memory.