Network access node virtual fabrics configured dynamically over an underlay network
Network access node virtual fabrics configured dynamically over an underlay network are described. A centralized controller, such as a software-defined networking (SDN) controller, of a packet switched network is configured to establish one or more virtual fabrics as overlay networks on top of the physical underlay network of the packet switched network. For example, the SDN controller may define multiple sets of two of more access nodes connected to the packet switched network, and the access nodes of a given one of the sets may use a new data transmission protocol, referred to generally herein as a fabric control protocol (FCP), to dynamically setup tunnels as a virtual fabric over the packet switched network. The FCP tunnels may include all or a subset of the parallel data paths through the packet switched network between the access nodes for a given virtual fabric.
1. A network system comprising:
a plurality of servers;
a packet switched network comprising a centralized controller; and
a plurality of access nodes, each of the access nodes coupled to the packet switched network, wherein the plurality of access nodes includes a first access node coupled to a source server included within the plurality of servers and a second access node coupled to a destination server included within the plurality of servers, wherein the first access node and the second access node are each executing a Fabric Control Protocol (FCP),
wherein the centralized controller is configured to establish one or more virtual fabrics, wherein each of the virtual fabrics includes two or more of the access nodes, and wherein, to establish a first virtual fabric including the first access node and the second access node, the centralized controller is configured to notify each of the first access node and the second access node of other access nodes included in the first virtual fabric, and, in response, the first access node and the second access node are configured to establish a logical tunnel over a plurality of parallel data paths through the packet switched network between the first access node and the second access node,
wherein, when communicating a packet flow of packets between the source server and the destination server, the first access node is configured to:
send an FCP request message for an amount of data to be transferred in a packet flow of packets from the source server to the destination server, and
in response to receipt of an FCP grant message indicating an amount of bandwidth reserved for the packet flow, encapsulate the packets within payloads of FCP packets and send the FCP packets over the logical tunnel of the first virtual fabric by spraying the FCP packets of the packet flow across the plurality of parallel data paths through the packet switched network to the second access node in accordance with the reserved bandwidth by directing each of the FCP packets to one of the parallel data paths selected based on bandwidth characteristics of the one of the parallel data paths, and wherein the second access node is configured to:
in response to receipt of the FCP request message, perform grant scheduling and send the FCP grant message indicating the amount of bandwidth reserved for the packet flow, and
in response to receiving the FCP packets of the packet flow from the first access node, extract the packets encapsulated within the FCP packets and deliver the packets to the destination server.
2. The network system of claim 1 , wherein, to establish the first virtual fabric, the centralized controller is configured to define a set of two or more access nodes for the first virtual fabric that includes at least the first access node and the second access node, and notify each of the access nodes within the defined set of the other access nodes included in the defined set.
3. The network system of claim 2 , wherein, to establish the first virtual fabric, the access nodes within the defined set each executing the FCP are configured to establish logical tunnels with the other access nodes included in the defined set as the first virtual fabric over the packet switched network, wherein the logical tunnels of the first virtual fabric include the plurality of parallel data paths through the packet switched network between the access nodes within the defined set.
4. The network system of claim 1 , wherein, to deliver the packets to the destination server, the second access node is configured to reorder the packets into an original sequence of the packet flow and deliver the reordered packets to the destination server.
5. The network system of claim 1 , wherein the centralized controller is configured to reset the access nodes for one of the virtual fabrics without impacting the other virtual fabrics established over the packet switched network.
6. The network system of claim 1 , wherein the centralized controller is configured to exchange different security parameters for each of the different virtual fabrics established over the packet switched network.
7. The network system of claim 6 , wherein the centralized controller is configured to distribute different encryption keys for each of the different virtual fabrics for use by the access nodes included in the virtual fabrics, such that only the access nodes for a given one of the virtual fabrics can decrypt packets exchanged over the given one of the virtual fabrics.
8. The network system of claim 1 , wherein each of the virtual fabrics comprises an overlay network and the packet switched network comprises an underlay network.
9. The network system of claim 1 , wherein, to spray the FCP packets of the packet flow across the plurality of parallel data paths, the first access node is configured to direct each of the FCP packets to a randomly, pseudo-randomly, or round-robin selected one of the parallel data paths.
10. The network system of claim 1 , wherein, to spray the FCP packets of the packet flow across the plurality of parallel data paths, the first access node is configured to direct each of the FCP packets to a least loaded one of the parallel data paths selected based on a byte count per path.
11. The network system of claim 1 , wherein, to spray the FCP packets of the packet flow across the plurality of parallel data paths, the first access node is configured to direct each of the FCP packets to a weighted randomly selected one of the parallel data paths in proportion to available bandwidth in the one of the virtual fabrics.
12. The network system of claim 1 , wherein the access nodes for the one of the virtual fabrics are configured to provide full mesh connectivity through the packet switched network between any pairwise combination of the servers coupled to the access nodes for the one of the virtual fabrics.
13. The network system of claim 1 , wherein the first access node has full mesh connectivity to a subset of the access nodes included in a logical rack as a first-level network fanout, and wherein the first one of the access nodes is configured to spray the FCP packets of the packet flow across the first-level network fanout to the subset of the access nodes included in the logical rack.
14. The network system of claim 13 , wherein each of the access nodes has full mesh connectivity to a subset of core switches included in the packet switched network as a second-level network fanout, and wherein each of the subset of access nodes included in the logical rack is configured to spray the FCP packets of the packet flow across the second-level network fanout to the subset of core switches.
15. The network system of claim 1 , wherein the packet switched network comprises a routing and switching fabric of one or more data centers, a local area network (LAN), a wide area network (WAN), or a collection of one or more networks.
16. A method comprising:
interconnecting a plurality of servers by a packet switched network and a plurality of access nodes, each of the access nodes coupled to the packet switched network, wherein the plurality of access nodes includes a first access node coupled to a source server included within the plurality of servers and a second access node coupled to a destination server included within the plurality of servers, wherein the first access node and the second access node are each executing a Fabric Control Protocol (FCP);
establishing, by a centralized controller of the packet switched network, one or more virtual fabrics, wherein each of the virtual fabrics includes two or more of the access nodes, wherein establishing a first virtual fabric including the first access node and the second access node comprises:
notifying, by the centralized controller, each of the first access node and the second access node of other access nodes included in the first virtual fabric, and
in response, establishing, by the first access node and the second access node, a logical tunnel over a plurality of parallel data paths through the packet switched network between the first access node and the second access node; and
communicating a packet flow of packets between the source server and the destination server including:
sending, by the first access node, an FCP request message for an amount of data to be transferred in a packet flow of packets from the source server to the destination server,
in response to receipt of the FCP request message, performing, by the second access node, grant scheduling and sending, by the second access node, an FCP grant message indicating the amount of bandwidth reserved for the packet flow,
in response to receipt of the FCP grant message indicating the amount of bandwidth reserved for the packet flow, encapsulating, by the first access node, the packets within payloads of FCP packets,
sending, by the first access node, the FCP packets over the logical tunnel of the first virtual fabric by spraying, by the first access node, the FCP packets of the packet flow across the plurality of parallel data paths through the packet switched network to the second access node in accordance with the reserved bandwidth by directing each of the FCP packets to one of the parallel data paths selected based on bandwidth characteristics of the one of the parallel data paths, and
in response to receiving the FCP packets of the packet flow from the first access node, extracting, by the second access node, the packets encapsulated within the FCP packets and delivering, by the second access node, the packets to the destination server.
17. The method of claim 16 , wherein establishing the first virtual fabric comprises defining, by the centralized controller, a set of two or more access nodes for the first virtual fabric that includes at least the first access node and the second access node, and notifying each of the access nodes within the defined set of the other access nodes included in the defined set.
18. The method of claim 17 , wherein establishing the first virtual fabric further comprises establishing, by the access nodes within the defined set each executing the FCP, logical tunnels with the other access nodes included in the defined set as the first virtual fabric over the packet switched network, wherein the logical tunnels of the first virtual fabric include the plurality of parallel data paths through the packet switched network between the access nodes within the defined set.
19. The method of claim 16 , wherein, to deliver the packets to the destination server, the second access node is configured to reorder the packets into an original sequence of the packet flow and deliver the reordered packets to the destination server.
20. The method of claim 16 , wherein the first access node has full mesh connectivity to a subset of the access nodes included in a logical rack as a first-level network fanout, the method further comprising spraying, by the first one of the access nodes, the FCP packets of the packet flow across the first-level network fanout to the subset of the access nodes included in the logical rack.
21. The method of claim 20 , wherein each of the access nodes has full mesh connectivity to a subset of core switches included in the packet switched network as a second-level network fanout, the method further comprising spraying, by each of the subset of access nodes included in the logical rack, the FCP packets of the packet flow across the second-level network fanout to the subset of core switches.