Coalescing messages using a network interface controller
View Patent ↗An injection descriptor corresponding to a destination node may be stored in memory. A network interface controller (NIC) may determine that one or more messages added to the injection descriptor are to be transmitted to the destination node. The NIC may then lock the injection descriptor so that no additional message can be added to the injection descriptor, and the NIC may load the one or more messages. The NIC may then generate a network packet that includes the one or more messages, and the NIC may transmit the network packet to the destination node.
1. A computer implemented method for coalescing messages, the method comprising:
storing, in a memory, an injection descriptor, the injection descriptor corresponding to a destination node;
determining, by a network interface controller (NIC), that two or more messages added to the injection descriptor are to be transmitted to the destination node, the NIC having direct memory access (DMA) to the memory;
locking, by the NIC, the injection descriptor to prevent an additional message from being added to the injection descriptor;
loading, by the NIC, the two or more messages;
clearing, by the NIC and after loading the two or more messages, the injection descriptor;
unlocking, by the NIC and after clearing the injection descriptor, the injection descriptor;
determining whether the network is busy;
in response to determining that the network is busy, generating, by the NIC, a network packet, the network packet including the two or more messages, and transmitting the network packet to the destination node; and
in response to determining that the network is not busy, generating, by the NIC, a set of network packets, the set of network packets including a separate packet for each of the two or more messages, and transmitting, by the NIC, the set of network packets to the destination node.
2. The method of claim 1 , wherein the generating, by the NIC, the network packet comprises coalescing the two or more messages.
3. The method of claim 1 , the method further comprising:
generating, by an application, a message to be sent to the destination node; and
adding, by the application, the message to the injection descriptor.
4. The method of claim 3 , the method further comprising:
determining, by the application, that the message is the only message added to the injection descriptor; and
adding, by the application, a pointer identifying the injection descriptor to an injection queue served by the NIC.
5. The method of claim 3 , wherein the adding, by the application, the message to the injection descriptor comprises using a compare-and-swap (CAS) instruction.
6. The method of claim 3 , wherein the adding, by the application, the message to the injection descriptor comprises using a load-link/store-conditional (LL/SC) instruction set.
7. The method of claim 1 , wherein the locking the injection descriptor comprises setting a flag in the memory, the flag indicating that the NIC is accessing the injection descriptor.
8. The method of claim 1 , wherein the locking the injection descriptor comprises using a store-like atomic memory operation (AMO).
9. A system for coalescing messages, the system comprising:
a memory having an injection descriptor, the injection descriptor corresponding to a destination node;
a network interface controller (NIC), wherein the NIC is configured to perform a method comprising:
determining that two or more messages added to the injection descriptor are to be transmitted to the destination node, the NIC having direct memory access (DMA) to the memory;
locking the injection descriptor to prevent an additional message from being added to the injection descriptor;
loading the two or more messages;
clearing, after loading the two or more messages, the injection descriptor;
unlocking, after clearing the injection descriptor, the injection descriptor;
determining whether the network is busy;
in response to determining that the network is busy, generating a network packet, the network packet including the two or more messages, and transmitting the network packet to the destination node; and
in response to determining that the network is not busy, generating a network packet for each of the two or more messages, and transmitting the two or more network packets to the destination node.
10. The system of claim 9 , wherein the generating the network packet comprises coalescing the two or more messages.
11. The system of claim 9 , wherein the system further includes a processor, the processor being configured to:
generate a message to be sent to the destination node; and
add the message to the injection descriptor.
12. The system of claim 11 , wherein the processor is further configured to:
determine that the message is the only message added to the injection descriptor; and
add a pointer identifying the injection descriptor to an injection queue served by the NIC.
13. The system of claim 11 , wherein the processor is configured to add the message to the injection descriptor using a compare-and-swap (CAS) instruction.
14. The system of claim 11 , wherein the processor is configured to add the message to the injection descriptor using a load-link/store-conditional (LL/SC) instruction set.
15. A computer program product for coalescing messages, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instruction executable by a network interface controller (NIC) to cause the NIC to perform a method comprising:
storing, in a memory, an injection descriptor, the injection descriptor corresponding to a destination node;
determining that two or more messages added to the injection descriptor are to be transmitted to the destination node, the NIC having direct memory access (DMA) to the memory;
locking the injection descriptor to prevent an additional message from being added to the injection descriptor;
loading the two or more messages;
clearing, after loading the two or more messages, the injection descriptor;
unlocking, after clearing the injection descriptor, the injection descriptor;
determining whether the network is busy;
in response to determining that the network is busy, generating a network packet, the network packet including the two or more messages, and transmitting the network packet to the destination node; and
in response to determining that the network is not busy, generating a network packet for each of the two or more messages, and transmitting the two or more network packets to the destination node.
16. The computer program product of claim 15 , wherein the generating the network packet comprises coalescing the two or more messages.
17. The computer program product of claim 15 , wherein the locking the injection descriptor comprises setting a flag in the memory, the flag indicating that the NIC is accessing the injection descriptor.
18. The computer program product of claim 15 , wherein the locking the injection descriptor comprises using a store-like atomic memory operation (AMO).