System and method for user space NVMe-over-RDMA host library
A method, computer program product, and computer system for exposing, by a user block layer of a computing device, a storage device as a block device to an application associated with an IO request. A Non-Volatile Memory Express (NVMe) protocol layer of the computing device may expose a NVMe storage device associated with the IO request to the user block layer. A NVMe-over-RDMA layer of the computing device may create a NVMe RDMA queue for the IO request to be processed.
1. A computer-implemented method comprising:
exposing, by a user block layer of a computing device, a storage device as a block device to an application associated with an IO request, wherein the user block layer includes an interface layer exposing the storage device to the application, exposing a unified API for all supported block device back ends, and implementing common mechanisms for block device back ends;
obtaining, by the user block layer, an IO descriptor list and a destination namespace associated with the IO request, wherein each IO descriptor of the IO descriptor list includes one or more of a command ID, and IO vector, and an op-code;
pre-registering, by the user block layer, application memory used for IO data associated with the IO request, wherein pre-registering the application memory occurs before processing the IO request and includes
pinning physical memory within the application memory and
preparing a virtual to physical address translation;
exposing, by a Non-Volatile Memory Express (NVMe) protocol layer of the computing device, a NVMe storage device associated with the IO request to the user block layer; and
creating, by a NVMe-over-Remote Direct Memory Access (RDMA) layer of the computing device, a NVMe RDMA queue for the IO request to be processed.
2. The computer-implemented method of claim 1 further comprising linking, by the NVMe protocol layer, a namespace to a controller to access the NVMe storage device.
3. The computer-implemented method of claim 1 wherein creating the NVMe RDMA queue includes creating, by the NVMe-over-RDMA layer, an RDMA queue.
4. The computer-implemented method of claim 3 further comprising destroying, by the NVMe-over-RDMA layer, the RDMA queue.
5. The computer-implemented method of claim 3 further comprising submitting, by the NVMe-over-RDMA layer, an RDMA work request associated with the IO request to the RDMA queue.
6. A computer program product residing on a non-transitory computer readable storage medium having a plurality of instructions stored thereon which, when executed across one or more processors, causes at least a portion of the one or more processors to perform operations comprising:
exposing, by a user block layer of a computing device, a storage device as a block device to an application associated with an IO request, wherein the user block layer includes an interface layer exposing the storage device to the application, exposing a unified API for all supported block device back ends, and implementing common mechanisms for block device back ends;
obtaining, by the user block layer, an IO descriptor list and a destination namespace associated with the IO request, wherein each IO descriptor of the IO descriptor list includes one or more of a command ID, and IO vector, and an op-code;
pre-registering, by the user block layer, application memory used for IO data associated with the IO request, wherein pre-registering the application memory occurs before processing the IO request and includes
pinning physical memory within the application memory and
preparing a virtual to physical address translation;
exposing, by a Non-Volatile Memory Express (NVMe) protocol layer of the computing device, a NVMe storage device associated with the IO request to the user block layer; and
creating, by a NVMe-over-Remote Direct Memory Access (RDMA) layer of the computing device, a NVMe RDMA queue for the IO request to be processed.
7. The computer program product of claim 6 wherein the operations further comprise linking, by the NVMe protocol layer, a namespace to a controller to access the NVMe storage device.
8. The computer program product of claim 6 wherein creating the NVMe RDMA queue includes creating, by the NVMe-over-RDMA layer, an RDMA queue.
9. The computer program product of claim 8 wherein the operations further comprise destroying, by the NVMe-over-RDMA layer, the RDMA queue.
10. The computer program product of claim 8 wherein the operations further comprise submitting, by the NVMe-over-RDMA layer, an RDMA work request associated with the IO request to the RDMA queue.
11. A computing system including one or more processors and one or more memories configured to perform operations comprising:
exposing, by a user block layer of a computing device, a storage device as a block device to an application associated with an IO request, wherein the user block layer includes an interface layer exposing the storage device to the application, exposing a unified API for all supported block device back ends, and implementing common mechanisms for block device back ends;
obtaining, by the user block layer, an IO descriptor list and a destination namespace associated with the IO request, wherein each IO descriptor of the IO descriptor list includes one or more of a command ID, and IO vector, and an op-code;
pre-registering, by the user block layer, application memory used for IO data associated with the IO request, wherein pre-registering the application memory occurs before processing the IO request and includes
pinning physical memory within the application memory and
preparing a virtual to physical address translation;
exposing, by a Non-Volatile Memory Express (NVMe) protocol layer of the computing device, a NVMe storage device associated with the IO request to the user block layer; and
creating, by a NVMe-over-Remote Direct Memory Access (RDMA) layer of the computing device, a NVMe RDMA queue for the IO request to be processed.
12. The computing system of claim 11 wherein the operations further comprise linking, by the NVMe protocol layer, a namespace to a controller to access the NVMe storage device.
13. The computing system of claim 11 wherein creating the NVMe RDMA queue includes creating, by the NVMe-over-RDMA layer, an RDMA queue.
14. The computing system of claim 13 wherein the operations further comprise destroying, by the NVMe-over-RDMA layer, the RDMA queue.
15. The computing system of claim 13 wherein the operations further comprise submitting, by the NVMe-over-RDMA layer, an RDMA work request associated with the IO request to the RDMA queue.