Low latency post-send
A system is described having one or more processing devices that prepare a work queue entry (WQE), transmit the WQE or provide a doorbell indication indicating the WQE is available, and, in response to determining an execution failure has occurred, perform at least one additional step to facilitate completion of an operation associated with the WQE.
1 . A system comprising one or more circuits to:
prepare a work queue entry (WQE);
transmit the WQE to a destination device by using a plurality of threads to write the WQE to memory of the destination device; and
in response to determining an execution failure has occurred, perform at least one step to facilitate completion of an operation associated with the WQE.
2 . The system according to claim 1 , wherein the WQE is written to memory prior to transmitting the WQE.
3 . The system according to claim 1 , wherein the execution failure comprises failing to receive a completion indication within a predetermined amount of time.
4 . The system according to claim 1 , wherein the execution failure comprises receiving a status indicating failure.
5 . The system according to claim 1 , wherein the one or more circuits are further to repeat transmitting the WQE to the destination device after determining the execution failure has occurred.
6 . The system according to claim 1 , wherein the at least one step to facilitate completion of the operation associated with the WQE comprises writing a doorbell record to memory and repeating the transmitting of the WQE to the destination device.
7 . The system of claim 1 , wherein the one or more circuits are further to write the WQE to memory.
8 . The system of claim 1 , wherein the at least one step to facilitate completion of the operation associated with the WQE comprises updating a doorbell record (DBR) and performing a memory barrier.
9 . The system of claim 8 , wherein the at least one step to facilitate completion of the operation associated with the WQE further comprises retransmitting the WQE to the destination device or providing a doorbell indication to the destination device indicating that the WQE is available.
10 . The system of claim 1 , wherein the one or more circuits are further to stamp a WQE buffer.
11 . The system of claim 10 , wherein the WQE is transmitted to the destination device in an absence of writing the WQE to memory.
12 . The system of claim 10 , wherein the WQE is transmitted to the destination device after writing the WQE to memory, wherein no memory barrier was performed prior to transmitting the WQE to the destination device.
13 . The system of claim 1 , wherein the at least one step to facilitate completion of the operation associated with the WQE comprises:
writing the WQE to memory;
performing a memory barrier; and
ringing a doorbell (DB).
14 . The system of claim 13 , wherein the at least one step further comprises updating a doorbell record (DBR) and performing a second memory barrier.
15 . The system of claim 14 , wherein updating the DBR comprises updating the DBR with a producer index.
16 . The system of claim 1 , wherein the one or more circuits are to repeat at least one of preparing the WQE and transmitting the WQE to the destination device multiple times before performing the at least one step to facilitate completion of the operation associated with the WQE.
17 . The system of claim 1 , wherein the execution failure comprises receiving one or more of a completion indication comprising an entry in a completion queue (CQ), a status in a memory location, an asynchronous event, and an interrupt.
18 . The system of claim 17 , wherein receiving the completion indication comprises polling the CQ.
19 . The system of claim 18 , wherein the one or more circuits perform fewer than two memory barriers prior to polling the CQ.
20 . The system of claim 1 , wherein sequential WQEs are written to a single address of the memory of the destination device.
21 . The system of claim 1 , wherein sequential WQEs are written to different addresses of the memory of the destination device.
22 . The system of claim 1 , wherein the one or more circuits are further to synchronize the plurality of threads after preparing the WQE and before transmitting the WQE.
23 . The system of claim 1 , wherein prior to preparing the WQE, the one or more circuits determine a number of in-flight WQEs is less than a threshold.
24 . A communication apparatus comprising one or more circuits to:
prepare a work queue entry (WQE);
transmit the WQE to a destination device by using a plurality of threads to write the WQE to memory of the destination device; and
in response to determining an execution failure has occurred, perform at least one step to facilitate completion of an operation associated with the WQE.