Distributed switchless interconnect
A distributed switchless system characterized by full mesh connectivity is disclosed. The full mesh distributed switchless system allows direct and indirect communication between a source node and a destination node. In direct communication, data propagates via links connecting the source and destination nodes. In indirect communication, data is first sent to an intermediate node via links connecting the source and intermediate nodes. The intermediate node sends the data to the destination node via links connecting the intermediate node and the destination node. The traffic can be divided into all available links across the nodes, rather than only the links connecting the source and destination nodes. Because indirect communication uses more links compared to direct communication, the traffic in each link is smaller. Consequently, the switchless distributed interconnect system can operate with fewer links between any two nodes and links with smaller bandwidth.
1. A method of processing data in a full mesh distributed switchless interconnect system comprising:
receiving by a first processing engine in a full mesh distributed switchless interconnect system including a plurality of processing engines wherein each of the processing engines in the interconnect system includes a direct switchless connection to each other processing engine, first data indicating a first destination processing engine;
determining by the first processing engine that the first processing engine is the first destination processing engine in the full mesh distributed switchless interconnect system and processing the first data by the first processing engine based on determining that the first processing engine is the first destination processing engine;
receiving, by the first processing engine, second data indicating a second destination processing engine;
determining by the first processing engine that the first processing engine is not the second destination processing engine;
evaluating an availability of the direct switchless connection between the first processing engine and the second destination processing engine in the full mesh distributed switchless interconnect system; and
transmitting the second data to an intermediate processing engine based on a result of the evaluation of the availability of the direct switchless connection indicating the direct switchless connection is not available.
2. The method of claim 1 , wherein the first data comprises at least one master cell which comprises at least one packet.
3. The method of claim 2 , wherein the at least one master cell contains a header with a queue sequence number.
4. The method of claim 3 , wherein processing the first data comprises:
ordering the at least one master cell based on the queue sequence number;
extracting the at least one packet; and
sending the at least one packet to a processing unit coupled to the first processing engine.
5. The method of claim 1 , further comprising forming by the first processing engine at least one master cell comprising at least one packet.
6. The method of claim 5 , further comprising determining a size of the master cell based on at least one of a link capacity, a processing engine capacity, a number of links, and a number of intermediate processing engines.
7. The method of claim 5 , wherein forming comprises adding packets to the at least one master cell until a timeout parameter expires.
8. The method of claim 5 , wherein forming comprises adding the at least one packet to the at least one master cell based on an amount of available packets and an urgency of delivery.
9. The method of claim 1 , further comprising
forming by the first processing engine a plurality of master cells from third data received from a processing unit coupled to the first processing engine, wherein the plurality of master cells contains different packets; and
transmitting the plurality of master cells to a plurality of intermediate processing engines.
10. The method of claim 1 , further comprising:
broadcasting by the first processing engine that a first link coupling the first processing engine to a second processing engine is inactive, if the first processing engine fails to receive a keepalive message from the second processing engine.
11. The method of claim 1 , further comprising:
sending by the first processing engine a first keepalive message to a second processing engine;
broadcasting by the first processing engine that a first link coupling the first processing engine to the second processing engine is inactive, if the first processing engine fails to receive a message from the second processing engine in response to the first keepalive message.
12. The method of claim 1 , further comprising:
sending by the first processing engine a first keepalive message to a second processing engine;
broadcasting by the first processing engine that a first link coupling the first processing engine to the second processing engine is inactive, if the first processing engine receives a message from the second processing engine indicating a failure to receive the first keepalive message.
13. In a full mesh distributed switchless interconnect system, an apparatus comprising:
a processing unit; and
a first processing engine coupled to the processing unit and having a direct switchless connection to each other processing engine of a plurality of other processing engines comprising the full mesh distributed switchless interconnect system, wherein the first processing engine is configured to receive via the full mesh distributed switchless interconnect system first data indicating a first destination processing engine;
wherein the first processing engine is configured to determine that the first processing engine is the first destination processing engine and to deliver the first data to the processing unit based on the determination that the first processing engine is the first destination processing engine; and
wherein the first processing engine is further configured to:
receive via the full mesh distributed switchless interconnect system second data indicating a second destination processing engine,
determine that the first processing engine is not the second destination processing engine,
evaluate an availability of the direct switchless connection between the first processing engine and the second destination processing engine in the full mesh distributed switchless interconnect system, and
transmit the data to an intermediate processing engine in the full mesh distributed switchless interconnect system based on a result of the evaluation of the availability of the direct switchless connection between the first processing engine and the second destination processing engine.
14. The apparatus of claim 13 , wherein the first processing engine comprises an assembly unit configured to form at least one master cell comprising at least one packet based on third data received from the processing unit.
15. The apparatus of claim 14 , wherein the first processing engine is configured to transmit the at least one master cell to a plurality of intermediate processing engines.
16. The apparatus of claim 13 , wherein the first processing engine comprises a re-ordering unit configured to reorder at least one master cell received from a second processing engine and de-assemble the received master cell to at least one packet.
17. The apparatus of claim 13 , wherein:
the first processing engine is configured to broadcast that a first link coupling a second processing engine to the first processing engine is inactive if the first processing engine fails to receive a keepalive message from the second processing engine.
18. The apparatus of claim 13 , wherein:
the first processing engine is configured to transmit a first keepalive message to a second processing engine;
the first processing engine is configured to broadcast that a first link coupling the first processing engine to the second processing engine is inactive if the first processing engine fails to receive a message from the second processing engine in response to the first keepalive message.
19. The apparatus of claim 13 , wherein:
the first processing engine is configured to transmit a first keepalive message to a second processing engine;
the first processing engine is configured to broadcast that a first link coupling the first processing engine to the second processing engine is inactive if the first processing engine receives a message from the second processing engine indicating a failure to receive the first keepalive message.
20. The method of claim 1 , further comprising:
receiving by the first processing engine a plurality of packets from a processing unit coupled to the first processing engine, each packet being associated with a different destination processing engine; and
for each packet of the plurality of packets:
determine whether the packet exceeds an available length of a master cell associated with the destination processing engine associated with the packet,
when the packet does not exceed the available length, adding the packet to the master cell, and
when the packet does exceed the available length, transmitting the master cell, building a new master cell, and adding at least a portion of the packet to the new master cell.
21. The apparatus of claim 13 , wherein the first processing engine is further configured to:
receive from the processing unit a plurality of packets, each packet being associated with a different destination processing engine; and
for each packet of the plurality of packets:
determine whether the packet exceeds an available length of a master cell associated with the destination processing engine associated with the packet,
when the packet does not exceed the available length, add the packet to the master cell, and
when the packet does exceed the available length, transmit the master cell, build a new master cell, and add at least a portion of the packet to the new master cell.