Granted Patent B2

US 12,278,763 · App. 17/303,901 · Granted Apr 15, 2025

Inventors: Srihari Raju Vegesna (San Jose, CA); Narendra Jayawant Gathoo (San Jose, CA); Pradeep Sindhu (Los Altos Hills, CA); Jean-Marc Frailong (Rancho Mirage, CA); Gopesh Goyal (Cupertino, CA); Suresh Vemula (Milpitas, CA); John David Huber (San Diego, CA); Chetan Ambalal Shah (Milpitas, CA)

Assignee: Microsoft Technology Licensing, LLC

H04L47/12H04L47/11H04L47/27H04L49/90

Abstract

Claims (65)

1. A network system comprising:

a plurality of servers including a source server and a destination server;

a network fabric comprising a plurality of core switches; and

a plurality of data processing units (DPUs) including a source DPU coupled to the source server and a destination DPU coupled to the destination server, wherein the source DPU and the destination DPU are configured to establish a logical tunnel over a plurality of parallel data paths across the core switches included within the network fabric between the source DPU and the destination DPU,

wherein a sender node of the source DPU is configured to:

spray packets of a packet flow from an ingress interface of the source DPU across the plurality of parallel data paths of the logical tunnel to an egress interface of the destination DPU by directing each of the packets to one of the parallel data paths, and

modify a send window size based on a degree of congestion at the egress interface of the destination DPU, wherein one or more of the plurality of parallel data paths are identified as failed paths, and wherein the degree of congestion is determined based on a number of the one or more failed paths.

2. The network system of claim 1 , wherein the sender node of the source DPU is configured to spray the packets of the packet flow to the receiver node of the destination DPU until an amount of data in flight is equal to the send window size.

3. The network system of claim 1 , wherein the sender node of the source DPU is configured to determine the degree of congestion at the egress interface of the destination DPU.

4. The network system of claim 3 , wherein to determine the degree of congestion, the sender node of the source DPU is configured to:

measure round-trip latency from the source DPU to the destination DPU and back to the source DPU; and

determine the degree of congestion at the egress interface of the destination DPU based on the round-trip latency or round-trip latency variation.

5. The network system of claim 3 , wherein to determine the degree of congestion, the sender node of the source DPU is configured to receive a message from a receiver node of the destination DPU indicating the degree of congestion at the egress interface of the destination DPU.

6. The network system of claim 1 , wherein a receiver node of the destination DPU is configured to:

determine the degree of congestion at the egress interface of the destination DPU; and

send a message to the sender node indicating the degree of congestion at the egress interface.

7. The network system of claim 6 , wherein to determine the degree of congestion, the receiver node of the destination DPU is configured to:

measure an amount of bandwidth at the egress interface of the destination DPU; and

determine the degree of congestion at the egress interface based on a comparison of the measured amount of bandwidth to a maximum bandwidth for the egress interface of the destination DPU.

8. The network system of claim 6 , wherein to determine the degree of congestion, the receiver node of the destination DPU is configured to:

measure a length of a queue maintained at the receiver node of the destination DPU to receive the packets from the sender node of the source DPU; and

determine the degree of congestion at the egress interface of the destination DPU based on the length of the queue increasing.

9. The network system of claim 6 , wherein to determine the degree of congestion, the receiver node of the destination DPU is configured to:

measure one-way latency from the source DPU to the destination DPU; and

determine the degree of congestion at the egress interface of the destination DPU based on the one-way latency or one-way latency variation.

10. The network system of claim 6 , wherein the source DPU and the destination DPU are each executing one of a fabric control protocol (FCP) or a reliable fabric control protocol (rFCP) extension, and

wherein, to send the message indicating the degree of congestion at the egress interface, the receiver node of the destination DPU is configured to send an FCP grant message or an rFCP grant/ACK message to the sender node including congestion control information.

11. The network system of claim 1 , wherein, in response to an indication of packet loss, the sender node sets the send window size equal to a loss window size and sets a configurable threshold to be less than a maximum send window size, and wherein the indication of packet loss comprises one of a negative acknowledgment (NACK) from a receiver node of the destination DPU or a retransmit timeout at the sender node.

12. The network system of claim 1 , wherein the degree of congestion is indicated by an amount of explicit congestion notification (ECN) marked bytes received by a receiver node of the destination DPU, and wherein the sender node of the source DPU reduces the send window size based on a ratio of the amount of ECN marked bytes to a total amount of bytes acknowledged by the receiver node of the destination DPU.

13. The network system of claim 1 ,

wherein the sender node of the source DPU includes a transmit queue length value in each packet of the packet flow that is sprayed to a receiver node of the destination DPU;

wherein the receiver node computes a window scale down factor based on a ratio of a total sender transmit queue length of all sender nodes and the transmit queue length received from the sender node; and

wherein the degree of congestion is indicated by the window scale down factor, and wherein the sender node reduces its maximum send window size based on the window scale down factor.

14. The network system of claim 1 , wherein a receiver node of the destination DPU sets a congestion status flag included in a message sent to the sender node of the source DPU based on one or more of a receiver buffer limit being reached, a receiver reorder table limit being reached, a detection of loss of multiple packets in a given window, or a total sender transmit queue length being greater than a threshold.

15. The network system of claim 14 , wherein, in response to receipt of the set congestion status flag, the sender node of the source DPU sets the send window size to an initial window size, and transitions from an unsolicited mode to a request-grant mode of data transmission.

16. The network system of claim 1 , wherein a receiver node of the destination DPU reorders the packets of the packet flow, and wherein the receiver node drops out of order packets once a reorder database utilization level reaches a deadlock avoidance threshold.

17. The network system of claim 1 ,

wherein the sender node of the source DPU has full mesh connectivity to a subset of DPUs included in a logical rack as a first-level network fanout, and wherein the sender node is configured to spray the packets of the packet flow across the first-level network fanout to the subset of the DPUs included in the logical rack; and

wherein each of the DPUs has full mesh connectivity to a subset of the core switches as a multi-level network fanout, and wherein each of the subset of the DPUs included in the logical rack is configured to spray the packets of the packet flow across the multi-level network fanout to the subset of the core switches.

18. A method comprising:

establishing a logical tunnel over a plurality of parallel data paths between a source data processing unit (DPU) and a destination DPU connected by a network fabric having a plurality of core switches, wherein the source DPU is coupled to a source server and the destination DPU is coupled to a destination server;

spraying, by a sender node of the source DPU, packets of a packet flow from an ingress interface of the source DPU across the plurality of parallel data paths of the logical tunnel to an egress interface of the destination DPU by directing each of the packets to one of the parallel data paths; and

modifying, by the sender node of the source DPU, a send window size based on a degree of congestion at the egress interface of the destination DPU, wherein one or more of the plurality of parallel data paths are identified as failed paths, and wherein the degree of congestion is determined based on a number of the one or more failed paths.

19. The method of claim 18 , further comprising determining, by the sender node of the source DPU, the degree of congestion at the egress interface of the destination DPU.

20. The method of claim 19 , wherein determining the degree of congestion comprises:

measuring, by the sender node of the source DPU, round-trip latency from the source DPU to the destination DPU and back to the source DPU; and

determining the degree of congestion at the egress interface of the destination DPU based on the round-trip latency or round-trip latency variation.

21. The method of claim 19 , wherein determining the degree of congestion comprises receiving, by the sender node of the source DPU, a message from a receiver node of the destination DPU indicating the degree of congestion at the egress interface of the destination DPU.

22. The method of claim 18 , further comprising:

determining, by a receiver node of the destination DPU, the degree of congestion at the egress interface of the destination DPU; and

sending, by the receiver node of the destination DPU, a message to the sender node of the source DPU indicating the degree of congestion at the egress interface.

23. The method of claim 22 , wherein determining the degree of congestion comprises:

measuring, by the receiver node of the destination DPU, an amount of bandwidth at the egress interface of the destination DPU; and

determining the degree of congestion at the egress interface based on a comparison of the measured amount of bandwidth to a maximum bandwidth for the egress interface of the destination DPU.

24. The method of claim 22 , wherein determining the degree of congestion comprises:

measuring, by the receiver node of the destination DPU, a length of a queue maintained at the receiver node of the destination DPU to receive the packets from the sender node of the source DPU; and

determining the degree of congestion at the egress interface of the destination DPU based on the length of the queue increasing.

25. The method of claim 22 , wherein determining the degree of congestion comprises:

measuring, by the receiver node of the destination DPU, one-way latency from the source DPU to the destination DPU; and

determining the degree of congestion at the egress interface of the destination DPU based on the one-way latency or one-way latency variation.

26. The method of claim 22 , wherein the source DPU and the destination DPU are each executing one of a fabric control protocol (FCP) or a reliable fabric control protocol (rFCP) extension, and wherein sending the message indicating the degree of congestion at the egress interface comprises sending, by the receiver node of the destination DPU, an FCP grant message or an rFCP grant/ACK message to the sender node of the source DPU including congestion control information.

27. A computing system configured to:

establish a logical tunnel over a plurality of parallel data paths between a source data processing unit (DPU) and a destination DPU connected by a network fabric having a plurality of core switches, wherein the source DPU is coupled to a source server and the destination DPU is coupled to a destination server;

spray, by a sender node of the source DPU, packets of a packet flow from an ingress interface of the source DPU across the plurality of parallel data paths of the logical tunnel to an egress interface of the destination DPU by directing each of the packets to one of the parallel data paths; and

modify, by the sender node of the source DPU, a send window size based on a degree of congestion at the egress interface of the destination DPU, wherein one or more of the plurality of parallel data paths are identified as failed paths, and wherein the degree of congestion is determined based on a number of the one or more failed paths.

Assignments (6)

Continuity (6)

References Cited (400)

Cited By (3)

US 12,556,474 US 12,669,943 US 12,712,805