Dual-ported PCI express-based storage cartridge including single-ported storage controllers
A storage cartridge may include a storage controller comprising a single PCIe port and a PCIe switch. The PCIe switch may include a first PCIe port communicatively coupled to a first PCIe fabric, a second PCIe port communicatively coupled to a second, different PCIe fabric, and a third PCIe port communicatively coupled to the single PCIe port of the storage controller. The first PCIe port and the second PCIe port may be configured to be selectively communicatively coupled to a non-transparent bridge (NTB) of the PCIe switch.
1. A storage cartridge comprising:
a storage controller comprising a single peripheral component interconnect express (PCIe) port;
a PCIe switch comprising:
a first PCIe port configured to be communicatively coupled to a first PCIe fabric;
a second PCIe port configured to be communicatively coupled to a second, different PCIe fabric; and
a third PCIe port communicatively coupled to the single PCIe port of the storage controller, wherein the first PCIe port and the second PCIe port are configured to be selectively communicatively coupled to a non-transparent bridge (NTB) of the PCIe switch;
a first sideband control interface communicatively coupled to the storage controller and configured to allow the storage controller to communicate with a first external device while bypassing the PCIe switch;
a second sideband control interface communicatively coupled to the storage controller and configured to allow the storage controller to communicate with a second external device while bypassing the PCIe switch; and
a switch control interface communicatively coupling the storage controller to the PCIe switch, wherein the storage controller is configured to:
receive a NTB assign command from the first external device or the second external device; and
communicate a command based on the NTB assign command to the PCIe switch via the switch control interface to cause the NTB to communicatively couple to either the first PCIe port of the PCIe switch or the second PCIe port of the PCIe switch.
2. The storage cartridge of claim 1 , wherein:
the storage controller comprises a first storage controller;
the storage cartridge further comprises a second storage controller comprising a single PCIe port; and
the PCIe switch further comprises a fourth PCIe port communicatively coupled to the single PCIe port of the second storage controller.
3. The storage cartridge of claim 2 , further comprising a first storage blade and a second storage blade, wherein the first storage blade comprises the PCIe switch, the first storage controller, and a first plurality of non-volatile memory devices, and wherein the second storage blade comprises the second storage controller and a second plurality of non-volatile memory devices.
4. A network attached storage (NAS) system comprising:
a first control CPU;
a second control CPU;
a first peripheral component interconnect express (PCIe) switch communicatively coupled to the first control CPU;
a second PCIe switch communicatively coupled to the second control CPU; and
a storage cartridge, wherein the storage cartridge comprises:
a storage controller comprising a single PCIe port; and
a third PCIe switch comprising:
a first PCIe port communicatively coupled to the first PCIe switch;
a second PCIe port communicatively coupled to the second PCIe switch;
a third PCIe port communicatively coupled to the single PCIe port of the storage controller, wherein the first PCIe port and the second PCIe port are configured to be selectively communicatively coupled to a non-transparent bridge (NTB) of the third PCIe switch;
a first sideband control interface communicatively coupled to the storage controller and configured to allow the storage controller to communicate with the first control CPU while bypassing the third PCIe switch; and
a second sideband control interface communicatively coupled to the storage controller and configured to allow the storage controller to communicate with the second control CPU while bypassing the third PCIe switch; and
a switch control interface communicatively coupling the storage controller to the third PCIe switch, wherein the storage controller is configured to receive a NTB assign command from the first external device or the second external device; and communicate a command based on the NTB assign command to the third PCIe switch via the switch control interface to cause the NTB to communicatively couple to either the first PCIe port of the third PCIe switch or the second PCIe port of the third PCIe switch.
5. The NAS system of claim 4 , wherein:
the storage controller comprises a first storage controller;
the storage cartridge further comprises a second storage controller comprising a single PCIe port; and
the PCIe switch further comprises a fourth PCIe port communicatively coupled to the single PCIe port of the second storage controller.
6. The NAS system of claim 5 , wherein the storage cartridge further comprises a first storage blade and a second storage blade, wherein the first storage blade comprises the third PCIe switch, the first storage controller, and a first plurality of non-volatile memory devices, and wherein the second storage blade comprises the second storage controller and a second plurality of non-volatile memory devices.
7. The NAS system of claim 4 , wherein the storage cartridge comprises a plurality of storage cartridges, wherein each respective storage cartridge of the plurality of storage cartridges comprises:
a respective storage controller comprising a single PCIe port; and
a respective third PCIe switch comprising:
a respective first PCIe port communicatively coupled to the first PCIe switch;
a respective second PCIe port communicatively coupled to the second PCIe switch; and
a respective third PCIe port communicatively coupled to the single PCIe port of the respective storage controller, wherein the respective first PCIe port and the respective second PCIe port are configured to be selectively communicatively coupled to a respective non-transparent bridge (NTB) of the respective third PCIe switch.
8. A method comprising:
receiving, by a storage controller of a storage cartridge, a non-transparent bridge (NTB) assign command from a control CPU, wherein the storage controller comprises a single peripheral component interconnect express (PCIe) port, wherein the storage cartridge comprises the storage controller and a PCIe switch, and wherein the PCIe switch comprises a first PCIe port communicatively coupled to a first PCIe fabric, a second PCIe port communicatively coupled to a second, different PCIe fabric, and a third PCIe port communicatively coupled to the single PCIe port of the storage controller;
communicating, by the storage controller, to the PCIe switch, based on the NTB assign command, a command to cause the NTB to communicatively couple to either the first PCIe port of the PCIe switch or the second PCIe port of the PCIe switch, wherein:
the storage cartridge further comprises:
a first sideband control interface communicatively coupled to the storage controller and configured to allow the storage controller to communicate with the control CPU while bypassing the PCIe switch;
a second sideband control interface communicatively coupled to the storage controller and configured to allow the storage controller to communicate with a second external device while bypassing the PCIe switch; and
the storage controller receives the NTB assign command from the control CPU via the first sideband control interface, wherein:
the storage cartridge further comprises a switch control interface communicatively coupling the storage controller to the PCIe switch; and
the storage controller communicates the command based on the NTB assign command to the PCIe switch via the switch control interface to cause the NTB to communicatively couple to either the first PCIe port of the PCIe switch or the second PCIe port of the PCIe switch.
9. The method of claim 8 , wherein:
the storage controller comprises a first storage controller;
the storage cartridge further comprises a second storage controller comprising a single PCIe port; and
the PCIe switch further comprises a fourth PCIe port communicatively coupled to the single PCIe port of the second storage controller.