Data center network having optical permutors
A network system for a data center is described in which a switch fabric may provide full mesh interconnectivity such that any servers may communicate packet data to any other of the servers using any of a number of parallel data paths. Moreover, according to the techniques described herein, edge-positioned access nodes, optical permutation devices and core switches of the switch fabric may be configured and arranged in a way such that the parallel data paths provide single L2/L3 hop, full mesh interconnections between any pairwise combination of the access nodes, even in massive data centers having tens of thousands of servers. The plurality of optical permutation devices permute communications across the optical ports based on wavelength so as to provide, in some cases, full-mesh optical connectivity between edge-facing ports and core-facing ports.
1. A method comprising:
receiving packet-based communications over a set of input optical ports of an optical permutation device, wherein the optical permutation device comprises a plurality of optical chips, wherein the set of input optical ports couple the optical permutation device to a set of access nodes positioned between the optical permutation device and a set of servers within a packet-based network;
permuting, based on wavelength, the packet-based communications across a set of output optical ports of the optical permutation device, wherein permuting includes, for each of the plurality of optical chips:
receiving an optical signal from each of the input optical ports, wherein each of the optical signals includes a plurality of wavelengths,
splitting the plurality of wavelengths in each of the optical signals, and
forwarding the plurality of wavelengths carried by each of the optical signals to the output optical ports such that the output optical ports each carry a unique permutation of the input optical ports and the plurality of wavelengths and where no single output optical port carries multiple optical communications having a same wavelength; and
forwarding, by the optical permutation device and over the set of output optical ports, the unique permutations to a set of one or more core switches within the packet-based network.
2. The method of claim 1 ,
wherein the access nodes, the core switches, and the optical permutation device are configured to provide full mesh connectivity between any pairwise combination of the servers within the set of servers.
3. The method of claim 1 ,
wherein the access nodes, the core switches, and the optical permutation device are configured to connect any pairwise combination of the access nodes by at most a single layer three (L 3 ) hop.
4. The method of claim 1 ,
wherein the access nodes, the core switches, and the optical permutation device are configured to provide a plurality of parallel data paths between each pairwise combination of the access nodes.
5. The method of claim 4 ,
wherein the plurality of access nodes includes a first access node coupled to a source server and a second access node coupled to a destination server; and
wherein, when communicating a packet flow of packets between the source server and the destination server, the first access node sprays the packets of the packet flow across the plurality of parallel data paths to the second access node.
6. The method of claim 5 ,
wherein the second access node reorders the packets into an original sequence of the packet flow and delivers the reordered packets to the destination server.
7. The method of claim 5 ,
wherein the first access node sprays the packets of the packet flow across the plurality of parallel data paths by directing each of the packets to a round-robin selected one of the parallel data paths.
8. The method of claim 5 ,
wherein the first access node sprays the packets of the packet flow across the plurality of parallel data paths by directing each of the packets to one of the parallel data paths selected based on available bandwidth of the one of the parallel data paths.
9. A network system comprising:
a plurality of servers;
a switch fabric comprising a plurality of core switches;
a plurality of access nodes, each of the access nodes coupled to a subset of the servers to communicate data packets between the servers; and
an optical permutation device optically coupling the access nodes to the core switches by optical links to communicate the data packets between the access nodes and the core switches as optical signals, and wherein the optical permutation device is configured to receive packet-based communications over a set of input optical ports coupling the optical permutation device to the access nodes and permute, based on wavelength, the packet-based communications across a set of output optical ports of the optical permutation device; and
a plurality of optical chips, each included within the optical permutation device, wherein each of the optical chips is configured to:
receive an optical signal from each of the input optical ports, wherein each of the optical signals includes a plurality of wavelengths,
split the plurality of wavelengths in each of the optical signals,
forward the plurality of wavelengths carried by each of the optical signals to the output optical ports such that the output optical ports each carry a unique permutation of the input optical ports and the plurality of wavelengths so that each output optical port carries multiple optical communications at different wavelengths, and
enable forwarding, by the optical permutation device and over the set of output optical ports, the unique permutations to the core switches.
10. The network system of claim 9 ,
wherein the access nodes, the core switches, and the optical permutation device are configured to provide full mesh connectivity between any pairwise combination of the servers.
11. The network system of claim 9 ,
wherein the access nodes, the core switches, and the optical permutation device are configured to connect any pairwise combination of the access nodes by at most a single layer three (L 3 ) hop.
12. The network system of claim 9 ,
wherein the access nodes, the core switches, and the optical permutation device are configured to provide a plurality of parallel data paths between each pairwise combination of the access nodes.
13. The network system of claim 12 ,
wherein the plurality of access nodes includes a first access node coupled to a source server and a second access node coupled to a destination server; and
wherein, when communicating a packet flow of packets between the source server and the destination server, the first access node sprays the packets of the packet flow across the plurality of parallel data paths to the second access node.
14. The network system of claim 13 ,
wherein the second access node reorders the packets into an original sequence of the packet flow and delivers the reordered packets to the destination server.
15. The network system of claim 13 ,
wherein the first access node sprays the packets of the packet flow across the plurality of parallel data paths by directing each of the packets to a randomly selected one of the parallel data paths.
16. The network system of claim 13 ,
wherein the first access node sprays the packets of the packet flow across the plurality of parallel data paths by directing each of the packets to one of the parallel data paths selected based on available bandwidth of the one of the parallel data paths.
17. An optical permutation device comprising a set of input optical ports, a set of output optical ports, and a plurality of optical chips, wherein the optical permutation device is configured to:
receive packet-based communications over the set of input optical ports, wherein the set of input optical ports couple the optical permutation device to a set of access nodes positioned between the optical permutation device and a set of servers within a packet-based network;
permute, based on wavelength, the packet-based communications across the set of output optical ports, wherein permuting includes, for each of the plurality of optical chips included within the optical permutation device:
receiving an optical signal from each of the input optical ports, wherein each of the optical signals includes a plurality of wavelengths,
splitting the plurality of wavelengths in each of the optical signals, and
forwarding the plurality of wavelengths carried by each of the optical signals to the output optical ports such that the output optical ports each carry a different permutation of the input optical ports and the plurality of wavelengths so that each output optical port carries optical communications at different wavelengths; and
forward, over the set of output optical ports, the unique permutations to a set of one or more core switches within the packet-based network.
18. The optical permutation device of claim 17 ,
wherein the access nodes, the core switches, and the optical permutation device is configured to provide full mesh connectivity between any pairwise combination of the servers within the set of servers.
19. The optical permutation device of claim 17 ,
wherein the access nodes, the core switches, and the optical permutation device is configured to connect any pairwise combination of the access nodes by at most a single layer three (L 3 ) hop.
20. The optical permutation device of claim 17 ,
wherein the access nodes, the core switches, and the optical permutation device is configured to provide a plurality of parallel data paths between each pairwise combination of the access nodes.