Methods and systems for servicing data requests in a multi-node system
A method for processing requests includes receiving a request comprising a virtual address, wherein the request is associated with an application executing on an operating system, identifying, based on data specified in the request, a logical volume associated with the data, making a first determination, based on the logical volume, that the logical volume is managed by a remote node, generating a data layout request to receive a data layout of the data from the remote node, receiving the data layout, wherein the data layout specifies a first physical address on the remote node that stores the data, initiating a copy request to copy the data from the first physical address to a second physical address on a local node, generating, based on the copy request, a virtual-to-physical address mapping between the virtual address and the second physical address; and initiating processing the request using the virtual-to-physical address mapping.
1. A method for processing requests, comprising:
receiving, by a space manager, a request comprising a virtual address, wherein the request is associated with an application executing on an operating system and wherein the space manager is executing in a container on the operating system;
identifying, based on data specified in the request, a logical volume associated with the data;
making a first determination, based on the logical volume, that the logical volume is managed by a remote node;
generating a data layout request to receive a data layout of the data from the remote node;
receiving, from the remote node, the data layout, wherein the data layout specifies a first physical address on the remote node that stores the data;
initiating a copy request to copy the data from the first physical address to a second physical address on a local node;
generating, based on the copy request, a virtual-to-physical address mapping between the virtual address and the second physical address; and
initiating processing the request using the virtual-to-physical address mapping.
2. The method of claim 1 , wherein the application is executing on a second container on the operating system.
3. The method of claim 1 , wherein prior to initiating the copy request, the method further comprises:
identifying a sparse virtual space segment of a sparse virtual space of the remote node, wherein the sparse virtual space segment is associated with a memory segment and wherein the memory segment is associated with the first physical address; and
associating the sparse virtual space segment with the virtual address.
4. The method of claim 3 ,
wherein the first physical address is located on a Persistent Memory (PMem) device of the remote node,
wherein the second physical address is located on a PMem device of the local node.
5. The method of claim 4 , wherein the copy request specifies using Remote Direct Memory Access (RDMA) to copy the data from the first physical address to the second physical address.
6. The method of claim 1 , wherein the data layout request specifies a pool segment of a pool associated with the data.
7. The method of claim 6 , wherein the pool is a memory pool and the pool segment is a memory pool segment.
8. A non-transitory computer readable medium comprising instructions which, when executed by a computer processor, enables the computer processor to perform a method for processing requests, the method comprising:
receiving, by a space manager, a request comprising a virtual address, wherein the request is associated with an application executing on an operating system and wherein the space manager is executing in a container on the operating system;
identifying, based on data specified in the request, a logical volume associated with the data;
making a first determination, based on the logical volume, that the logical volume is managed by a remote node;
generating a data layout request to receive a data layout of the data from the remote node;
receiving, from the remote node, the data layout, wherein the data layout specifies a first physical address on the remote node that stores the data;
initiating a copy request to copy the data from the first physical address to a second physical address on a local node;
generating, based on the copy request, a virtual-to-physical address mapping between the virtual address and the second physical address; and
initiating processing the request using the virtual-to-physical address mapping.
9. The non-transitory computer readable medium of claim 8 , wherein the application is executing on a second container on the operating system.
10. The non-transitory computer readable medium of claim 8 , wherein prior to initiating the copy request, the method further comprises:
identifying a sparse virtual space segment of a sparse virtual space of the remote node, wherein the sparse virtual space segment is associated with a memory segment and wherein the memory segment is associated with the first physical address; and
associating the sparse virtual space segment with the virtual address.
11. The non-transitory computer readable medium of claim 10 ,
wherein the first physical address is located on a Persistent Memory (PMem) device of the remote node,
wherein the second physical address is located on a PMem device of the local node.
12. The non-transitory computer readable medium of claim 11 , wherein the copy request specifies using Remote Direct Memory Access (RDMA) to copy the data from the first physical address to the second physical address.
13. The non-transitory computer readable medium of claim 8 , wherein the data layout request specifies a pool segment of a pool associated with the data.
14. The non-transitory computer readable medium of claim 13 , wherein the pool is a memory pool and the pool segment is a memory pool segment.
15. A node, comprising:
memory;
persistent storage; and a processor, wherein the processor is configured to:
receive a request comprising a virtual address, wherein the request is associated with an application executing on an operating system;
identify, based on data specified in the request, a logical volume associated with the data; make a first determination, based on the logical volume, that the logical volume is managed by a remote node;
generate a data layout request to receive a data layout of the data from the remote node;
receive, from the remote node, the data layout, wherein the data layout specifies a first physical address on the remote node that stores the data;
initiate a copy request to copy the data from the first physical address to a second physical address on a local node;
generate, based on the copy request, a virtual-to-physical address mapping between the virtual address and the second physical address;
and initiating processing the request using the virtual-to-physical address mapping.
16. The node of claim 15 , wherein the application is executing on a second container on the operating system.
17. The node of claim 15 , wherein prior to initiating the copy request, the processor is further configured to:
identify a sparse virtual space segment of a sparse virtual space of the remote node, wherein the sparse virtual space segment is associated with a memory segment and wherein the memory segment is associated with the first physical address; and
associate the sparse virtual space segment with the virtual address.
18. The node of claim 17 ,
wherein the first physical address is located on a Persistent Memory (PMem) device of the remote node,
wherein the second physical address is located on a PMem device of the local node.
19. The node of claim 18 , wherein the copy request specifies using Remote Direct Memory Access (RDMA) to copy the data from the first physical address to the second physical address.
20. The node of claim 15 , wherein the data layout request specifies a pool segment of a pool associated with the data.