Method and apparatus for achieving dynamic capacity and high availability in multi-stage data networks using adaptive flow-based routing
Methods and systems for determining paths for flows within a multi-stage network made up of clusters of processing nodes. The flow paths may be determined without knowledge of whether or not packets of a particular flow will actually traverse specific ones of the clusters within the multi-stage network. In various implementations, the nodes of the multi-stage network may be coupled to one or more physical network switches through respective physical interfaces and a virtual connectivity grid superimposed thereon and configured through the use of a flow routing framework and system management framework to group the nodes into a number of clusters. The nodes of each cluster are configured to perform similar packet processing functions and the clusters are interconnected through virtual networks to which the nodes are communicatively coupled via virtual interfaces overlaid on top of the physical network interfaces.
1. A method, comprising determining, in a centralized fashion, paths for flows within a multi-stage network made up of processing nodes grouped into clusters having similar functionality, by encoding node selection information representing flow path decisions for all clusters of the multi-stage network in packets of each flow within the multi-stage network, the centralized determination being made by a designated node acting as an entry point for the flows into the multi-stage network and that sets up an end-to-end flow paths within the multi-stage network.
2. The method of claim 1 , wherein the paths are determined without knowledge of whether or not packets of a particular flow will actually traverse specific ones of the clusters within the multi-stage network.
3. The method of claim 2 , wherein the node selection information is encoded in the packets by replacing other header information in the packets with the node selection information.
4. The method of claim 3 , wherein original header information present prior to encoding of the node selection information is restored to the packets prior to packet processing by application programs executing at nodes of the multi-stage network.
5. The method of claim 2 , wherein the node selection information is encoded in the packets by appending the node selection information to the packets.
6. The method of claim 5 , wherein the node selection information is stripped prior to packet processing by application programs executing at nodes of the multi-stage network.
7. The method of claim 2 , wherein individual nodes of each cluster are selected for inclusion in the paths according to load balancing criteria.
8. The method of claim 7 , wherein the load balancing criteria comprises a round-robin selection methodology.
9. The method of claim 7 , wherein the load balancing criteria comprises a weighted round-robin selection methodology.
10. The method of claim 7 , wherein the load balancing criteria comprises real time information from the nodes of the multi-stage network as to current load conditions.
11. The method of claim 2 , wherein prior to determining the paths for the flows, the flows are classified using a set of information present in the packets and according to criteria established by a network administrator, and flow paths are assigned based on policies set by the network administrator.
12. A method, comprising determining, in a distributed fashion, paths for flows within a multi-stage network made up of processing nodes grouped into clusters having similar functionality, by encoding node selection information representing flow path decisions for all clusters of the multi-stage network in packets of each flow within the multi-stage network, wherein next hops in the paths for the flows are locally determined at each of the nodes as packets of the flows traverse the flow paths within the multi-stage network.
13. The method of claim 12 , wherein the actual flow routes do not include all of the clusters specified in the flow paths.
14. The method of claim 13 , wherein information regarding the actual flow routes is used in determining subsequent new flow paths.
15. The method of claim 12 , wherein the node selection information is encoded in the packets by replacing other header information in the packets with the node selection information.
16. The method of claim 12 , wherein original header information present prior to encoding of the node selection information is restored to packet headers prior to packet processing by application programs executing at the nodes of the multi-stage network.
17. The method of claim 12 , wherein node selection is made on the basis of applying a hash function to a flow identifier encoded within the packet.
18. The method of claim 17 , wherein the node selection information is appended to each packet.
19. The method of claim 18 , wherein the node selection information is stripped prior to packet processing by application programs executing at the nodes of the multi-stage network.
20. The method of claim 19 , wherein the nodes are grouped into various clusters, and individual nodes of each cluster are selected according to load balancing criteria.
21. The method of claim 20 , wherein the load balancing criteria comprises a round-robin selection methodology.