Providing a memory region or memory window access notification on a system area network
View Patent ↗Mechanisms for providing a memory region/memory window (MR/MW) access notification on a system area network are provided. Whenever a previously allocated MR/MW is accessed, such as via a remote direct memory access (RDMA) read/write operation, a notification of the access is generated and written to a queue data structure associated with the MR/MW. In one illustrative embodiment, this queue data structure may be a MR/MW event queue (EQ) data structure that is created and used for all consumer processes and all MR/MWs. In other illustrative embodiments, the EQ is associated with a protection domain. In yet another illustrative embodiment, an event record may be posted to an asynchronous event handler in response to the accessing of the MR/MW. In another illustrative embodiment, a previously posted queue element may be used to generate a completion queue element in response to the accessing of the MR/MW.
1. A method, in a data processing system, for informing a process of an access to a pre-allocated memory region/memory window (MR/MW) of the data processing system, comprising:
receiving, in a channel adapter of the data processing system, a Remote Direct Memory Access (RDMA) message, from an external computing device, for accessing a MR/MW of the data processing system identified by an identifier of the MR/MW included in the RDMA message;
processing the RDMA message in the channel adapter to write data associated with the RDMA message to the MR/MW identified by the identifier;
automatically notifying, by logic of the channel adapter of the data processing system, a process associated with the MR/MW, executing on the data processing system, of the write of data to the MR/MW in response to the writing of the data to the MR/MW, wherein the logic of the channel adapter of the data processing system performs the automatic notification without waiting for receipt of a completion queue entry message from the external computing device;
processing, by the process, the data in the MR/MW in response to the notification;
posting a work queue element (WQE) to a receive queue of the channel adapter; and
generating a completion queue element (CQE) in a completion queue of the channel adapter, based on information in the WQE, in response to receiving the RDMA message for accessing the MR/MW, wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW comprises processing, by the process, the CQE.
2. A method, in a data processing system, for informing a process of an access to a pre-allocated memory region/memory window (MR/MW) of the data processing system, comprising:
receiving, in a channel adapter, a Remote Direct Memory Access (RDMA) message for accessing a MR/MW of the data processing system identified by an identifier of the MR/MW included in the RDMA message;
processing the RDMA message in the channel adapter to write data associated with the RDMA message to the MR/MW identified by the identifier;
automatically notifying a process associated with the MR/MW of the write of data to the MR/MW in response to the writing of the data to the MR/MW;
processing, by the process, the data in the MR/MW in response to the notification, wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW comprises posting an event queue element (EQE) to an event queue (EQ);
posting a work queue element (WQE) to a receive queue of the channel adapter; and
generating a completion queue element (CQE) in a completion queue of the channel adapter, based on information in the WQE, in response to receiving the RDMA message for accessing the MR/MW, wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW comprises processing, by the process, the CQE.
3. The method of claim 2 , wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW further comprises:
generating an interrupt to an interrupt handler;
processing the interrupt to identify the process associated with the MR/MW based on the identifier; and
sending a notification to the process that an EQE is ready for processing in the EQ in response to processing the interrupt.
4. The method of claim 2 , wherein the EQ receives EQEs associated with write operations to all MR/MWs of the data processing system.
5. The method of claim 2 , wherein the EQ is associated with a protection domain of the data processing system such that there are a plurality of EQs, one for each protection domain of the data processing system.
6. The method of claim 5 , wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW further comprises:
identifying a protection domain associated with the MR/MW based on the identifier;
identifying an EQ associated with the identified protection domain; and
posting the EQE to the EQ associated with the identified protection domain.
7. The method of claim 1 , wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW comprises posting an event record to an asynchronous event handler of the data processing system.
8. The method of claim 7 , wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW further comprises:
identifying, by the asynchronous event handler, the process associated with the MR/MW based on the identifier; and
sending a notification to the process that the event record is available for processing in the asynchronous event handler.
9. The method of claim 1 , wherein the RDMA message for accessing the MR/MW is a RDMA message generated by an external device in response to a RDMA read work request transmitted to the external device by the data processing system.
10. A computer program product comprising a computer readable storage device having a computer readable program, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:
receive, in a channel adapter of the data processing system, a Remote Direct Memory Access (RDMA) message, from an external computing device, for accessing a memory region/memory window (MR/MW), of the data processing system, identified by an identifier of the MR/MW included in the RDMA message;
process the RDMA message in the channel adapter to write data associated with the RDMA message to the MR/MW identified by the identifier;
automatically notify, by logic of the channel adapter of the data processing system, a process associated with the MR/MW, executing on the data processing system, of the write of data to the MR/MW in response to the writing of the data to the MR/MW, wherein the logic of the channel adapter of the data processing system performs the automatic notification without waiting for receipt of a completion queue entry message from the external computing device;
process, by the process, the data in the MR/MW in response to the notification;
post a work queue element (WQE) to a receive queue of the channel adapter; and generate a completion queue element (CQE) in a completion queue of the channel adapter, based on information in the WQE, in response to receiving the RDMA message for accessing the MR/MW, wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW comprises processing, by the process, the CQE.
11. A computer program product comprising a computer readable storage device having a computer readable program, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:
receive, in a channel adapter, a Remote Direct Memory Access (RDMA) message for accessing a memory region/memory window (MR/MW), of the data processing system, identified by an identifier of the MR/MW included in the RDMA message;
process the RDMA message in the channel adapter to write data associated with the RDMA message to the MR/MW identified by the identifier;
automatically notify a process associated with the MR/MW of the write of data to the MR/MW in response to the writing of the data to the MR/MW;
process, by the process, the data in the MR/MW in response to the notification, wherein the computer readable program causes the data processing system to automatically notify the process associated with the MR/MW of the write of data to the MR/MW by posting an event queue element (EQE) to an event queue (EQ);
post a work queue element (WQE) to a receive queue of the channel adapter; and generate a completion queue element (CQE) in a completion queue of the channel adapter, based on information in the WQE, in response to receiving the RDMA message for accessing the MR/MW, wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW comprises processing, by the process, the CQE.
12. The computer program product of claim 11 , wherein the computer readable program further causes the data processing system to automatically notify the process associated with the MR/MW of the write of data to the MR/MW by:
generating an interrupt to an interrupt handler;
processing the interrupt to identify the process associated with the MR/MW based on the identifier; and
sending a notification to the process that an EQE is ready for processing in the EQ in response to processing the interrupt.
13. The computer program product of claim 12 , wherein the EQ receives
EQEs associated with write operations to all MR/MWs of the data processing system.
14. The computer program product of claim 11 , wherein the EQ is associated with a protection domain of the data processing system such that there are a plurality of EQs, one for each protection domain of the data processing system.
15. The computer program product of claim 14 , wherein the computer readable program further causes the data processing system to automatically notify the process associated with the MR/MW of the write of data to the MR/MW by:
identifying a protection domain associated with the MR/MW based on the identifier;
identifying an EQ associated with the identified protection domain; and
posting the EQE to the EQ associated with the identified protection domain.
16. The computer program product of claim 10 , wherein the computer readable program causes the data processing system to automatically notify the process associated with the MR/MW of the write of data to the MR/MW by posting an event record to an asynchronous event handler of the data processing system.
17. The computer program product of claim 16 , wherein the computer readable program further causes the data processing system to automatically notify the process associated with the MR/MW of the write of data to the MR/MW by:
identifying, by the asynchronous event handler, the process associated with the MR/MW based on the identifier; and
sending a notification to the process that the event record is available for processing in the asynchronous event handler.
18. A data processing system, comprising:
a processor;
a memory coupled to the processor, the memory comprising one or more memory regions/memory windows (MR/MWs); and
a channel adapter coupled to the processor and the memory, wherein the channel adapter:
receives a Remote Direct Memory Access (RDMA) message, from an external computing device, for accessing a MR/MW, of the data processing system, identified by an identifier of the MR/MW included in the RDMA message,
processes the RDMA message to write data associated with the RDMA message to the MR/MW identified by the identifier,
automatically notify a process, executing on the data processing system, associated with the MR/MW of the write of data to the MR/MW in response to the writing of the data to the MR/MW, wherein the process processes the data in the MR/MW in response to the notification, and wherein the channel adapter of the data processing system automatically notifies the process without waiting for receipt of a completion queue entry message from the external computing device,
post a work queue element (WQE) to a receive queue of the channel adapter, and
generate a completion queue element (CQE) in a completion queue of the channel adapter, based on information in the WQE, in response to receiving the RDMA message for accessing the MR/MW, wherein automatically notifying the process associated with the MR/MW of the write of data to the MR/MW comprises processing, by the process, the CQE.