Remote direct memory access in multi-tier memory systems
A memory system having memory components, a remote direct memory access (RDMA) network interface card (RNIC), and a host system, and configured to: allocate a page of virtual memory for an application; map the page of virtual memory to a page of physical memory in the memory components; instruct the RNIC to perform an RDMA operation; perform, during the RDMA operation, a data transfer between the page of physical memory in the plurality of memory components and a remote device that is connected via a computer network to the remote direct memory access network interface card; and at least for a duration of the data transfer, lock a mapping between the page of virtual memory and the page of physical memory in the memory components.
1. A computer system, comprising:
a host system, operatively coupled to a plurality of memory components and a remote device that is connected via computer network to the remote direct memory access network interface card; and
lock, at least for a duration of the data transfer, a mapping between the page of virtual memory and the page of physical memory in the plurality of memory components; and
a controller configured to perform predictive data movements within the plurality of memory components;
wherein the remote direct memory access network interface card implements single root input/output virtualization, including a physical function and a plurality of virtual function, and a remote direct memory access driver running in a hypervisor that controls the physical function of the remote direct memory access network interface card.
2. The computer system of claim 1 , wherein the host system is further configured to unlock the mapping between the page of virtual memory and the plurality of memory components after the remote direct memory access operation.
3. The computer system of claim 1 , wherein the host system is further configured to unlock the mapping between the page of virtual memory and the plurality of memory components after the data transfer.
4. The computer system of claim 3 , wherein the host system is further configured to detect a completion of the data transfer via a notification agent coupled to the application.
5. The computer system of claim 3 , wherein the host system is further configured to detect a completion of the data transfer via a notification agent coupled to an operating system in which the application is running.
6. The computer system of claim 5 , wherein the host system is further configured to detect the completion of the data transfer via a remote direct memory access driver running in the operating system.
7. The computer system of claim 6 , wherein the remote device is mounted as a remote volume in the operating system.
8. The computer system of claim 3 , wherein the host system is further configured to detect a completion of the data transfer via the hypervisor.
9. The computer system of claim 8 , wherein the host system is further configured to detect the completion of the data transfer via the remote direct memory access driver running in the hypervisor.
10. The computer system of claim 9 , wherein the hypervisor has a memory access driver that locks the mapping during of the data transfer.
11. The computer system of claim 10 , wherein the memory components includes a first memory and a second memory slower than the first memory; and the page of the physical memory is in the second memory.
12. The computer system of claim 11 , wherein the first memory is volatile dynamic random access memory and the second memory is non-volatile cross point memory.
13. The computer system of claim 11 , wherein the host system locks the mapping between the page of virtual memory and the page of the second memory in response to the remote direct memory access operation.
14. The computer system of claim 13 , wherein the host system includes a central processing unit (CPU) and a memory management unit; and the CPU is coupled to the remote direct memory access network interface card via a peripheral component interconnect express (PCIe) bus.
15. The computer system of claim 13 , wherein the controller comprises a data orchestrator implemented using a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC) to implement an artificial neural network that predicts the data movements using the artificial neural network.
16. A method, comprising:
allocating a page of virtual memory for an application;
mapping the page of virtual memory to a page of physical memory in a plurality of memory components;
instructing, by the application, a remote direct memory access network interface card to perform a remote direct memory access operation, wherein during the remote direct memory access operation, the remote direct memory access network interface card performs a data transfer between the page of physical memory in the plurality of memory components and a remote device that is connected via a computer network to the remote direct memory access network interface card;
locking, at least for a duration of the data transfer, the mapping between the page of virtual memory and the page of physical memory in the plurality of memory components;
performing, via a controller, predictive data movements within the plurality of memory components;
implementing, via the remote direct memory access network interface card, single root input/output virtualization, including a physical function and a plurality of virtual functions; and
controlling, via a remote direct memory access driver running in a hypervisor, the physical function of the remote direct memory access network interface card.
17. The method of claim 16 , further comprising, after the data transfer:
unlocking the mapping; and
swapping, based on a workload prediction, the page of virtual memory from a first memory in the plurality of memory components to a second memory in the plurality of memory components, the second memory being faster to the first memory.
18. A non-transitory computer storage medium storing instructions which, when executed by a computing system, cause the computing system to perform a method, the method comprising:
allocating a page of virtual memory for an application running in the computing system;
mapping the page of virtual memory to a page of physical memory in a plurality of memory components;
instructing, by the application, a remote direct memory access network interface card to perform a remote direct memory access operation, wherein during the remote direct memory access operation, the remote direct memory access network interface card performs a data transfer between the page of physical memory in the plurality of memory components and a remote device that is connected via a computer network to the remote direct memory access network interface card;
locking, at least for a duration of the data transfer, the mapping between the page of virtual memory and the page of physical memory in the plurality of memory components;
performing, via a controller, predictive data movements within the plurality of memory components;
implementing, via the remote direct memory access network interface card, single root input/output virtualization, including a physical function and a plurality of virtual functions; and
controlling, via a remote direct memory access driver running in a hypervisor, the physical function of the remote direct memory access network interface card.