Scalable communication fabric system
Systems, methods, apparatuses, and software for computing systems are provided herein. In one example, a system includes a plurality of first modules each having a Peripheral Component Interconnect Express (PCIe) interface and a processor, and a plurality of second modules each having a PCIe interface. PCIe switch circuitry is coupled to the PCIe interfaces of the first modules and the PCIe interfaces of the second modules, wherein the PCIe switch circuitry is configured to establish logical isolation in the PCIe switch circuitry between one or more first modules and one or more second modules. At least one processor instantiates access to the one or more second modules for the one or more first modules over at least the logical isolation in the PCIe switch circuitry.
1. A system, comprising:
a plurality of processing modules each having a Peripheral Component Interconnect Express (PCIe) interface and a processor;
a plurality of second modules each having a PCIe interface;
PCIe switch circuitry coupled to the PCIe interfaces of the processing modules and the PCIe interfaces of the second modules, wherein the PCIe switch circuitry is configured to establish logical isolation in the PCIe switch circuitry between one or more processing modules and one or more second modules; and
wherein at least one processor instantiates access to the one or more second modules for the one or more processing modules over at least the logical isolation in the PCIe switch circuitry.
2. The system of claim 1 , wherein first ones among the plurality of second modules comprise storage drives, and wherein second ones among the plurality of second modules comprise network interfaces.
3. The system of claim 2 , wherein first network interfaces are configured to interface with external devices coupled over associated network links, and wherein second network interfaces are configured to provide cluster interconnect over associated network links.
4. The system of claim 2 , wherein the network interfaces are configured to receive transactions over associated network links, and wherein the transactions are transferred by the second ones of the plurality of second modules over the PCIe switch circuitry using the logical isolation to corresponding processors among the processing modules.
5. The system of claim 4 , wherein the transactions comprise storage operations directed to storage drives among the second modules.
6. The system of claim 4 , wherein the network interfaces are configured to receive the transactions in an iSCSI protocol from external devices coupled over the associated network links.
7. The system of claim 1 , wherein one or more among the plurality of second modules comprise a storage drive.
8. The system of claim 7 , comprising:
at least one of the processing modules configured to receive a transaction into an associated processor, determine if the transaction corresponds to a subset of the second modules managed by the associated processor, and when the transaction does not correspond to the subset, then the associated processor responsively transfers the transaction over the PCIe switch circuitry to another of the processing modules having a processor that manages at least a target module among the second modules that corresponds to the transaction.
9. The system of claim 7 , comprising:
at least one of the processing modules configured to receive a transaction into an associated processor, determine if the transaction corresponds to a subset of the second modules managed by the associated processor, and when the transaction corresponds to the subset, then the associated processor responsively transfers the transaction over the PCIe switch circuitry to at least one of the second modules.
10. A method of operating a data storage system, comprising:
in Peripheral Component Interconnect Express (PCIe) switch circuitry communicatively coupled to PCIe interfaces of a plurality of processing modules and to PCIe interfaces of a plurality of second modules, establishing logical isolation in the PCIe switch circuitry between one or more processing modules each comprising a processor and one or more second modules; and
wherein at least one processor instantiates access to the one or more second modules by the one or more processing modules over at least the logical isolation in the PCIe switch circuitry.
11. The method of claim 10 , wherein first ones among the plurality of second modules comprise storage drives, and wherein second ones among the plurality of second modules comprise network interfaces.
12. The method of claim 11 , further comprising:
in first network interfaces, interfacing with external devices coupled over associated network links; and
in second network interfaces, providing cluster interconnect over associated network links.
13. The method of claim 11 , further comprising:
in the network interfaces, receiving transactions over associated network links, wherein the transactions are transferred by the second ones of the plurality of second modules over the PCIe switch circuitry using the logical isolation to corresponding processors among the processing modules.
14. The method of claim 13 , wherein the transactions comprise storage operations directed to storage drives among the second modules.
15. The method of claim 13 , further comprising:
in the network interfaces, receiving the transactions in an iSCSI protocol from external devices coupled over the associated network links.
16. The method of claim 10 , wherein one or more among the plurality of second modules comprise a storage drive.
17. The method of claim 16 , further comprising:
in at least one of the processing modules, receiving a transaction into an associated processor, and determining if the transaction corresponds to a subset of the second modules managed by the associated processor, wherein when the transaction does not correspond to the subset, the associated processor responsively transfers the transaction over the PCIe switch circuitry to another of the processing modules having a processor that manages at least one of the second modules that corresponds to the transaction.
18. The method of claim 16 , further comprising:
in at least one of the processing modules, receiving a transaction into an associated processor, determining if the transaction corresponds to a subset of the second modules managed by the associated processor, wherein when the transaction corresponds to the subset, the associated processor responsively transfers the transaction over the PCIe switch circuitry to at least one of the second modules.
19. A data system, comprising:
a plurality of processing modules each having a Peripheral Component Interconnect Express (PCIe) interface and a processor;
a plurality of network modules each having a PCIe interface and a network interface;
a plurality of third modules each having a PCIe interface;
PCIe switch circuitry coupled to the PCIe interfaces of the processing modules, the PCIe interfaces of the network modules, and the PCIe interfaces of the third modules, wherein the PCIe switch circuitry is configured to establish logical isolation in the PCIe switch circuitry between one or more processing modules, one or more network modules, and one or more third modules; and
wherein at least one processor instantiates access to the one or more third modules for the one or more processing modules over at least the logical isolation in the PCIe switch circuitry.
20. The data system of claim 19 , comprising:
the plurality of network modules configured to interface with external devices coupled over associated network links and receive storage operations over the associated network links directed to storage drives among the plurality of third modules; and
wherein the storage operations are transferred by the plurality of network modules over the PCIe switch circuitry using the logical isolation to corresponding processors among the plurality of processing modules.