IP Library › Granted Patent US 12,199,888
Granted Patent B2
US 12,199,888 · App. 18/425,968 · Granted Jan 14, 2025

Reliable transport offloaded to network devices

Inventors: Shaopeng He (Shanghai, CN); Cunming Liang (Shanghai, CN); Jiang Yu (Shanghai, CN); Ziye Yang (Shanghai, CN); Ping Yu (Shanghai, CN); Bo Cui (Shanghai, CN); Jingjing Wu (Shanghai, CN); Liang Ma (Shannon, IE); Hongjun Ni (Shanghai, CN); Zhiguo Wen (Shanghai, CN); Changpeng Liu (Shanghai, CN); Anjali Singhai Jain (Portland, OR); Daniel Daly (Santa Barbara, CA); Yadong Li (Portland, OR)
Assignee: Intel Corporation
H04L49/9057H04L1/1841H04L47/34H04L47/56H04L49/552H04L49/9094
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,199,888
App. No.
18/425,968
Granted
Jan 14, 2025
Kind
B2
Abstract

Examples described herein relate to offload reliable transport management to a network interface device and store packets to be resent, based on received packet receipt acknowledgements (ACKs), into one or more kernel space queues that are also accessible in user space.

Claims (39)

1. An apparatus comprising:

at least one processor;

at least one memory comprising instructions stored thereon, that if executed by the at least one processor, cause the at least one processor to:

offload reliable transport management to a network interface device, wherein the reliable transport management is based on quick User Datagram Protocol (UDP) Internet Connections (QUIC) and wherein the network interface device is to perform header split to copy a packet header and a packet payload to different destinations and

based on received packet receipt acknowledgements (ACKs), track packets to be resent, by access to one or more kernel space queues that are also accessible in user space.

2. The apparatus of claim 1 , wherein the at least one memory compris es instructions stored thereon, that if executed by the at least one processor, cause the at least one processor to:

identify descriptors available for reuse into the one or more kernel space queues that are also accessible in user space.

3. The apparatus of claim 1 , wherein content stored in the one or more kernel space queues are also accessible in user space without copying from memory accessible to user space to a memory accessible kernel space.

4. The apparatus of claim 1 , wherein:

a socket is to connect a service with a driver for the network interface device,

the one or more kernel space queues are associated with the socket, and

at least one of the one or more kernel space queues comprises a waiting queue for received packet receipt acknowledgements (ACKs).

5. The apparatus of claim 1 , wherein the one or more kernel space queues are based on an Address Family of an eXpress Data Path (AF_XDP) queue framework.

6. The apparatus of claim 1 , wherein:

at least one of the one or more kernel space queues is to identify one or more packets for which one or more ACKs have been received.

7. The apparatus of claim 1 , wherein the at least one processor is to execute a service that is to request transmission of at least one packet that is subject to receipt of an ACK.

8. The apparatus of claim 1 , comprising the network interface device to transmit at least one packet based on a request from the at least one processor.

9. A non-transitory computer-readable medium comprising instructions stored thereon, that if executed by one or more processors, cause the one or more processors to:

offload reliable transport management to a network interface device, wherein the reliable transport management is based on quick User Datagram Protocol (UDP) Internet Connections (QUIC) and wherein the network interface device is to perform header split to copy a packet header and a packet payload to different destinations.

10. The non-transitory computer-readable medium of claim 9 , comprising instructions stored thereon, that if executed by one or more processors, cause the one or more processors to:

track packets to be resent, based on received packet receipt acknowledgements (ACKs), into one or more kernel space queues that are also accessible in user space, wherein the one or more kernel space queues that are also accessible in user space are to identify descriptors available for reuse.

11. The non-transitory computer-readable medium of claim 10 , wherein:

a socket is to connect a service with a driver for the network interface device,

the one or more kernel space queues are associated with the socket, and

at least one of the one or more kernel space queues comprises a waiting queue for received packet receipt acknowledgements (ACKs).

12. The non-transitory computer-readable medium of claim 10 , wherein:

the one or more kernel space queues are based on an Address Family of an express Data Path (AF_XDP) queue framework.

13. An apparatus comprising:

a network interface device comprising circuitry to perform offloaded reliable transport for packet transmission based on a queue framework comprising one or more kernel space queues that are also accessible in user space, wherein the reliable transport is based on quick User Datagram Protocol (UDP) Internet Connections (QUIC) and wherein the circuitry is to perform header split to copy a packet header and a packet payload to different destinations.

14. The apparatus of claim 13 , wherein the one or more kernel space queues that are also accessible in user space are to track packets to be resent, based on received packet receipt acknowledgements (ACKs).

15. The apparatus of claim 13 , wherein:

the network interface device is to re-transmit a packet based on non-receipt of an acknowledgement (ACK) of packet receipt.

16. The apparatus of claim 13 , wherein:

at least one of the one or more kernel space queues is to identify at least one available descriptor that is available to associated with a received packet.

17. The apparatus of claim 13 , wherein the network interface device comprises circuitry to:

based on receipt of a packet having a sequence number corresponding to a non-received packet sequence number, provide access to one or more packet payloads associated with one or more received packet sequence numbers.

18. The apparatus of claim 13 , wherein the network interface device comprises one or more of: a network interface controller (NIC), a smart NIC, an infrastructure processing unit (IPU), and/or a data processing unit (DPU).

19. The apparatus of claim 13 , comprising a server comprising at least one processor that is to execute a service that is to request transmission of at least one packet that is subject to receipt of an acknowledgement (ACK).

20. The apparatus of claim 13 , comprising a server comprising at least one processor that is to execute a service that is to process at least one payload of at least one packet associated with a received packet sequence number.

Continuity (4)
Continuation 17981255 · Nov 4, 2022
Continuation 17238893 · Apr 23, 2021
Provisional Application 63145327 · Feb 3, 2021
Related Publication 20240283756A1 · Aug 22, 2024
References Cited (71)
US 8839370B2 · Singh et al. · 2014 [cited by applicant]
US 10552274B1 · Zhao et al. · 2020 [cited by applicant]
US 20110299409A1 · Vobbilisetty · 2011 [cited by examiner]
US 20150180736A1 · Leung · 2015 [cited by applicant]
US 20180295045A1 · Bali et al. · 2018 [cited by applicant]
US 20190044812A1 · Loftus et al. · 2019 [cited by applicant]
US 20190044994A1 · Sarangam et al. · 2019 [cited by applicant]
US 20190190891A1 · Pillai et al. · 2019 [cited by applicant]
US 20200053064A1 · Oprisan et al. · 2020 [cited by applicant]
US 20200099628A1 · Parikh et al. · 2020 [cited by applicant]
US 20200133795A1 · Rhodes et al. · 2020 [cited by applicant]
US 20200236140A1 · Srinivasan et al. · 2020 [cited by applicant]
US 20200287813A1 · Kutch et al. · 2020 [cited by applicant]
US 20200319812A1 · He et al. · 2020 [cited by applicant]
US 20210103403A1 · He et al. · 2021 [cited by applicant]
US 20210117242A1 · Groenendaal et al. · 2021 [cited by applicant]
US 20210243247A1 · He · 2021 [cited by applicant]
US 20210266253A1 · He · 2021 [cited by applicant]
US 20210194828A1 · He et al. · 2021 [cited by applicant]
US 20210352044A1 · Asveren · 2021 [cited by examiner]
US 20220029929A1 · Jain · 2022 [cited by applicant]
US 20220103530A1 · Daly · 2022 [cited by applicant]
CN 112513988A · 2021 [cited by applicant]
WO 2020030311A1 · 2020 [cited by applicant]
WO 2020041569A1 · 2020 [cited by applicant]
WO 2022125211A1 · 2022 [cited by applicant]
WO 2022169519A1 · 2022 [cited by applicant]
WO 2022225639A1 · 2022 [cited by applicant]
An Pratt and Keir Fraser, Arsenic: A User-Accessible Gigabit Ethernet Interface, year 2001, IEEE, 0-7803-7016-3/01, pp. 67-76 (Year: 2001). [cited by examiner]
Marcelo Abranches et al., ‘A Userspace Transport Stack Doesn't Have to Mean Losing Linux Processing’, 2020 IEEE Conference on Network Function Virtualization and Softwme Defined Networks (NFV-SDN), Dec. 24, 2020. [cited by examiner]
U.S. Appl. No. 17/238,893, filed Apr. 23, 2021, He et al. [cited by applicant]
“HTTP and gRPC Transcoding”, AIP-127, https://google.aip.dev/127, Last updated Aug. 22, 2019, 3 pages. [cited by applicant]
“Improving Network Monitoring and Management with Programmable Data Planes”, ONF, https://opennetworking.org/news-and-events/blog/improving-network-monitoring-and-management-with-programmable-data-planes/, Sep. 25, 2015… [cited by applicant]
“In-band Network Telemetry (INT) Dataplane Specification”, Version 2.1, The P4.org Applications Working Group, Nov. 11, 2020, 56 pages. [cited by applicant]
“Intel® Ethernet Adaptive Virtual Function (AVF) Hardware Architecture Specification (HAS)”, Intel Networking Division, Revision: 1.0, Feb. 2018, 90 pages. [cited by applicant]
“Introduction to Intel Ethernet Flow Director and Memcached Performance”, Intel White Paper, Oct. 13, 2014, 6 pages. [cited by applicant]
“Networking architecture”, Security Guide Docmentation, https://docs.openstack.org/security-guide/networking/architecture.html, Jul. 27, 2021, 6 pages. [cited by applicant]
“OpenStack Networking Guide”, openstack, https://docs.openstack.org/mitaka/networking-guide/, updated Aug. 16, 2019, 3 pages. [cited by applicant]
“P4-16 Language Specification”, version 1.2.1, The P4 Language Consortium, Jun. 11, 2020, 113 pages. [cited by applicant]
“Transcoding HTTP/JSON to gRPC”, Google Cloud Endpoints, cloud.google.com/endpoints/docs/grpc/transcoding, downloaded from the internet Jul. 28, 2021, 14 pages. [cited by applicant]
“Trellis”, Overview, Trellis is an open-source multi-purpose L2/L3 leaf-spine switching fabric., https://docs.trellisfabric.org/1.12/index.html, © Copyright 2019, Open Networking Foundation, 4 pages. [cited by applicant]
Bosshart, Pat, et al., “P4: programming protocol-independent packet processors”, ACM SIGCOMM Computer Communication Review vol. 44 Issue 3, https://dl.acm.org/doi/pdf/10.1145/2656877.2656890, published Jul. 28, 2014, 9 … [cited by applicant]
Colyer, Adrian, “Snap: a microkernel approach to host networking”, the morning paper, https://blog.acolyer.org/2019/11/11/snap-networking/, Nov. 11, 2019, 6 pages. [cited by applicant]
Crilly, Liam, “Introducing a Technology Preview of NGINX Support for QUIC and HTTP/3”, NGINX, https://www.nginx.com/blog/introducing-technology-preview-nginx-support-for-quic-http-3/, Jun. 10, 2020, 4 pages. [cited by applicant]
Diptanu Gon Choudhury, ‘XDP-Programmable Data Path in the Linux Kernel’, ;login: Spring 2018, vol. 43, No. 1, pp. 26-30, 2018, 6 pages. [cited by applicant]
First Office Action for U.S. Appl. No. 17/981,255, Mailed May 24, 2023, 7 pages. [cited by applicant]
Ian Pratt and Keir Fraser, Arsenic: A User-Accessible Gigabit Ethernet Interface, year 2001, IEEE, 0-7803-7016-3/01, pp. 67-76 (Year: 2001). [cited by applicant]
Intel, “Intel Ethernet Adaptive Virtual Function (AVF) Hardware Architecture Specification (HAS)”, Networking Division, Revision: 1.0, Feb. 2018, 90 pages. [cited by applicant]
Intel, “Introduction to Intel® Ethernet Flow Director and Memcached Performance”, White Paper, Intel® Ethernet Flow Director and Memcached Performance, Copyright © 2014 Intel Corporation, 6 pages. [cited by applicant]
International Search Report and Written Opinion for PCT Patent Application No. PCT/US21/57130, Mailed Feb. 21, 2022, 10 pages. [cited by applicant]
International Search Report and Written Opinion for PCT Patent Application No. PCT/US21/65177, Mailed Apr. 25, 2022, 10 pages. [cited by applicant]
International Search Report and Written Opinion for PCT Patent Application No. PCT/US22/11045, Mailed Apr. 25, 2022, 11 pages. [cited by applicant]
International Search Report and Written Opinion for PCT Patent Application No. PCT/US22/21795, Mailed Jun. 24, 2022, 12 pages. [cited by applicant]
Irfan, Talal, “AF_XDP101L Getting Started With an In-Kernel network stack Bypass”, Published in Emumba, https://blog.emumba.com/af-xdp-101-getting-started-with-an-in-kernel-network-stack-bypass-6e6901a285c8, Jun. 24, 20… [cited by applicant]
Iyengar, Ed, et. al., “QUIC: A UDP-Based Multiplexed and Secure Transport draft-ietf-quic-transport-29”, QUIC Standards Track, https://datatracker.ietf.org/doc/html/draft-ietf-quic-transport-29#page-23, Jun. 10, 2020, 1… [cited by applicant]
Karlsson Magnus, et al., ‘The path to DPDK speeds for AF XDP’, Linux Plumbers Conference, 2018, 9 pages. [cited by applicant]
Karlsson, Magnus , “Performance I40E 64-Byte Packets”, Fast Packet Processing in Linux with AF_XDP, FOSDEM 2018, 1 page. [cited by applicant]
Li Xing, Alibaba, et. al., “Hyperscale rte_flow toward productoin is really happening” DPDK Data Pane Development Kit, DPDK Summit Bordeaux 2019, 25 pages. [cited by applicant]
Marcelo Abranches, et. al., ‘A Userspace Transport Stack Doesn't Have to Mean Losing Linux Processing’, 2020 IEEE Conference on Nettwork Function Virtualization and Software Defined Networks (NFV-SDN), Dec. 24, 2020, 7 … [cited by applicant]
Mvs, Janakiram, “Service Mesh—The New Battleground For the Platfor Wars”, Sep. 20, 2020, 8 pages. [cited by applicant]
Nguyen Van Tu, et al., ‘Accelerating Virtual Network Functions with Fast-Slow Path Architecture using express Data Path’, IEEE Transactions on Network and Service Management vol. 17. Issue 3, Jun. 5, 2020, 13 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/238,893, Mailed Feb. 9, 2022, 10 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/238,893, Mailed Jun. 13, 2022, 8 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/981,255, Mailed Oct. 27, 2023, 8 pages. [cited by applicant]
P4.org, “Improving Network Monitoring and Management with Progammable Data Planes”, ONF, https://opennetworking.org/news-and-events/blog/improving-network-monitoring-and-management-with-programmable-data-planes/, Sep. 2… [cited by applicant]
Peterson, Larry, “Trellis”, Dashboard / CORD—Wiki Home / Project and Domains of Use, https://wiki.opecord.org/display/CORD/Trellis, Jul. 27, 2021, 3 pages. [cited by applicant]
Postel, J., “User Datagram Protocol”, RFC 768, Aug. 28, 1980, 4 pages. [cited by applicant]
Wiggins, Adam, “The Twelve-Factor App”, VII. Port binding, https://12factor.net/port-binding, Last updated 2017, 1 page. [cited by applicant]
Xing, Li, et al., “Hyperscale rte_flow toward production is really happening”, DPDK https://dpdkbordeaux2019.sched.com/event/RmZG, Sep. 20, 2019, 25 pages. [cited by applicant]
Zhang, Jiao, et al., “Fast Switch-Based Load Balancer Considering Application Server States”, IEEE, https://ieeexplore.ieee.org/document/9061132. [cited by applicant]
Extended European Search Report for Patent Application No. 21925082.6, Mailed Nov. 25, 2024, 14 pages. [cited by applicant]