Multiprocessor node controller circuit and method
View Patent ↗A multiprocessor computer system includes a first and second node controller, a number of processors being connected to each node controller, a memory connected to each node controller, a first input/output system connected to the first node controller, and a communications network connected between the node controllers. The first node controller includes: a crossbar unit to which are connected a memory port, an input/output port, a network port, and a plurality of independent processor ports. A first and a second processor port is connected between the crossbar unit and a first subset and a second subset, respectively, of the processors. The memory port includes a memory data port including a memory data bus and a memory address bus coupled to the first subset of memory chips, and a directory data port including a directory data bus and a directory address bus coupled to the second subset of memory chips.
1. A node controller in a computer system, comprising:
a datapath crossbar operable to pass message frames in a dedicated format;
a plurality of output queues, each output queue associated with a local source, a particular output queue operable to buffer outgoing message frames from the local source for transfer to the datapath crossbar, the particular output queue operable to rate match between a bandwidth of the datapath crossbar and a bandwidth of the local source, the particular output queue operable to provide synchronization between the datapath crossbar and the local source, the particular output queue having a unit interface customized to a unique requirement of the associated local source, the particular output queue having a datapath interface that is uniform with the datapath crossbar for all output queues;
a memory/directory interface unit operable to convert data to and from a format of a memory device coupled thereto and the dedicated format of the datapath crossbar;
an input/output interface unit operable to convert data to and from a format of a device coupled thereto and the dedicated format of the datapath crossbar;
a network interface unit operable to convert data to and from a network format and the dedicated format of the datapath crossbar;
a processing device interface unit operable to convert data to and from a format of a processing device coupled thereto and the dedicated format of the datapath crossbar;
wherein the output queues associated with the memory/directory interface unit include separate header and data buffers.
2. The node controller of claim 1 , wherein the unit interface operates at a clock speed of the local source and the datapath operates at a clock speed of the datapath crossbar.
3. The node controller of claim 1 , wherein the particular output queue is operable to bypass synchronization when the associated local source is synchronized with the datapath crossbar.
4. The node controller of claim 1 , wherein the dedicated format of a particular message frame indicates which port of the datapath crossbar the particular message frame is destined, whether the particular message is a request or a reply, and whether the particular message frame is a last frame of a message.
5. The node controller of claim 1 , wherein the datapath crossbar is operable to interleave message frames associated with different virtual channels.
6. The node controller of claim 1 , further comprising:
an arbiter operable to control message frame transfer within the datapath crossbar.
7. The node controller of claim 6 , wherein the arbiter prevents mixing of message frames for different messages on one virtual channel.
8. The node controller of claim 6 , wherein the arbiter is operable to provide wavefront arbitration for the output queues, wavefront arbitration allowing a particular output queue to read data upon grant from a buffer location in a first clock cycle and provide the data to the datapath crossbar in a second clock cycle.
9. The node controller of claim 6 , wherein the arbiter is operable to provide bypass arbitration, bypass arbitration allowing a particular output queue to provide data upon grant to the datapath crossbar in a single clock cycle upon receipt without storing the data in a buffer.
10. The node controller of claim 1 , wherein the header buffer handles all message frames associated with a request from the memory/directory interface unit.
11. The node controller of claim 1 , wherein the data buffer includes dual buffers written into simultaneously by the memory/directory interface unit and read alternatingly by the datapath crossbar.
12. The node controller of claim 1 , wherein the output queues associated with the memory directory unit include a dedicated output queue for the processing device interface unit and a dedicated output queue associated with the other units of the node controller.
13. The node controller of claim 1 , wherein the network interface unit is operable to provide message frames associated with read requests directly to the memory/directory interface unit and bypass the datapath crossbar.
14. The node controller of claim 1 , wherein the network interface unit and the memory/directory interface unit are aware of a buffer depth for its associated output queues.
15. The node controller of claim 14 , wherein the network interface unit and the memory directory interface unit are each operable to receive an indication that a buffer location in its associated output queues has become available for use.
16. The node controller of claim 1 , wherein the datapath crossbar is operable to pass copies of message frames entering and leaving the datapath crossbar unit to the network interface unit regardless of a destination for the message frames.
17. The node controller of claim 1 , further comprising:
a plurality of input queues associated with local destinations that are asynchronous with the datapath crossbar.
18. The node controller of claim 17 , wherein a fill level of the input queues is used to control a flow of message frames from the output queues.