Apparatus and method for buffer management for receive segment coalescing
Packets received non-contiguously from a network are processed by a network interface controller by coalescing received packet payload into receive buffers on a receive buffer queue and writing descriptors associated with the receive buffers for a same flow consecutively in a receive completion queue. System performance is optimized by reusing a small working set of provisioned receive buffers to minimize the memory footprint of memory allocated to store packet data. The remainder of the provisioned buffers are in an overflow queue and can be assigned to the network interface controller if the small working set of receive buffers is not sufficient to keep up with the received packet rate. The receive buffer queue can be refilled based on either timers or when the number of buffers in the receive buffer queue is below a configurable low watermark.
1. An apparatus comprising:
a network interface controller to couple to a memory, the memory to store a receive buffer queue comprising at least one receive buffer and a receive completion queue comprising at least one entry, the network interface controller to:
store received packet payloads for a Receive Segment Coalescing context in receive buffers on the receive buffer queue; and
write descriptors associated with all the receive buffers associated with the Receive Segment Coalescing context in consecutive entries in the receive completion queue after all the received packet payloads for the Receive Segment Coalescing context have been stored in the receive buffer queue.
2. The apparatus of claim 1 , wherein the Receive Segment Coalescing context is one of a Transmission Control Protocol/Internet Protocol (TCP/IP) flow or a QUIC transport protocol flow.
3. The apparatus of claim 1 , wherein the network interface controller and the memory are in a data center server.
4. The apparatus of claim 1 , wherein received packet payloads for the Receive Segment Coalescing context are stored in non-contiguous buffers in the receive buffer queue.
5. The apparatus of claim 1 , further comprising:
at least one processor core to:
upon receiving notification that received packet payloads for the Receive Segment Coalescing context are ready to be processed, to access consecutive descriptors in the receive completion queue to access data stored in the memory.
6. The apparatus of claim 1 , wherein the network interface controller to:
store received packet payloads in receive buffers on the receive buffer queue; and
write a current buffer index to a fixed location in memory to provide an indication of use of receive buffers on the receive buffer queue.
7. The apparatus of claim 1 , wherein the receive buffers are processed contiguously for the receive segment coalescing context.
8. A method comprising:
storing, in a memory, a receive buffer queue comprising at least one receive buffer and a receive completion queue comprising at least one entry;
storing received packet payloads for a Receive Segment Coalescing context in receive buffers in the receive buffer queue; and
writing descriptors associated with all the receive buffers for associated with the Receive Segment Coalescing context in consecutive entries in the receive completion queue after all the received packet payloads for the Receive Segment Coalescing context have been stored in the receive buffer queue.
9. The method of claim 8 , wherein the Receive Segment Coalescing context is one of a Transmission Control Protocol/Internet Protocol (TCP/IP) flow or a QUIC transport protocol flow.
10. The method of claim 8 , wherein received packet payloads for the Receive Segment Coalescing context are stored in non-contiguous buffers in the receive buffer queue.
11. The method of claim 8 , further comprising:
upon receiving notification that received packet payloads for the Receive Segment Coalescing context are ready to be processed, accessing consecutive descriptors in the receive completion queue to access data stored in the memory.
12. The method of claim 8 , further comprising:
storing, by a network interface controller, received packet payloads in receive buffers on the receive buffer queue; and
writing, by a network interface controller, a current buffer index to a fixed location in memory to provide an indication of use of receive buffers on the receive buffer queue.
13. A system comprising:
a memory module, the memory module comprising at least one volatile memory integrated circuit, the volatile memory integrated circuit to store a receive buffer queue comprising at least one receive buffer and a receive completion queue comprising at least one entry; and
a network interface controller to couple to the memory module, the network interface controller to:
store received packet payloads for a Receive Segment Coalescing context in receive buffers on the receive buffer queue; and
write descriptors associated with all the receive buffers associated with the Receive Segment Coalescing context in consecutive entries in the receive completion queue after all the received packet payloads for the Receive Segment Coalescing context have been stored in the receive buffer queue.
14. The system of claim 13 , wherein the Receive Segment Coalescing context is one of a Transmission Control Protocol/Internet Protocol (TCP/IP) flow or a QUIC transport protocol flow.
15. The system of claim 13 , further comprising:
at least one processor core to:
upon receiving notification that packets are ready to be processed, to access consecutive descriptors in the receive completion queue to access data stored in the memory module.
16. The system of claim 15 , wherein the at least one processor core to:
allocate a working set of receive buffers to the receive buffer queue to minimize an amount of memory provisioned for the receive buffer queue;
refill the receive buffer queue, using receive buffers returned from the receive completion queue; and
allocate a buffer to the receive buffer queue from an overflow queue if a buffer is not available in the working set of receive buffers.
17. The system of claim 16 , wherein the network interface controller to:
store received packet payloads in receive buffers on the receive buffer queue; and
write a current buffer index to a fixed location in memory to provide an indication of use of receive buffers on the receive buffer queue.
18. The system of claim 15 , wherein the receive buffers are processed contiguously for the receive segment coalescing context.
19. A non-transitory machine-readable medium for storing machine-executable instructions that, cause a system to:
store, in a memory, a receive buffer queue comprising at least one receive buffer and a receive completion queue comprising at least one entry;
store received packet payloads for a Receive Segment Coalescing context in receive buffers on the receive buffer queue; and
write descriptors associated with all the receive buffers associated with the Receive Segment Coalescing context in consecutive entries in the receive completion queue after all the received packet payloads for the Receive Segment Coalescing context have been stored in the receive buffer queue.
20. The non-transitory machine-readable medium of claim 19 , wherein received packet payloads for the Receive Segment Coalescing context are stored in non-contiguous buffers in the receive buffer queue.
21. The non-transitory machine-readable medium of claim 19 , further comprising:
upon receiving notification that packets are ready to be processed, access consecutive descriptors in the receive completion queue to access data stored in the memory.
22. The non-transitory machine-readable medium of claim 19 , further comprising:
store, by a network interface controller, received packet payloads in receive buffers on the receive buffer queue; and
write, by a network interface controller, a current buffer index to a fixed location in memory to provide an indication of use of receive buffers on the receive buffer queue.