High bandwidth symmetrical storage controller
Provided herein are systems, apparatuses and methods (i.e., utilities) that allow for increasing the bandwidth of a processing complex of a storage controller. The utilities utilize a symmetrical approach where PCIe switches overcome limitations of prior art processor complexes. The symmetrical approach provided by the disclosed utilities as incorporated into a storage controller provides equal access from any host path/channel to any drive path/channel (i.e., storage element). More specifically, a first or a first set of PCIe switches connect front-end PCIe host bus adaptors, which are connectable to host systems, to front-end data paths of a plurality of PCIe memory controllers. A second or second set of PCIe switches connect backend host bus adapters, which are connectable to storage elements, to back-end data paths of the plurality of PCIe memory controllers. The symmetrical architecture provides at least twice the bandwidth of prior art architectures.
1. A storage controller, comprising:
four PCIe memory controllers, each having a PCIe front-end path, a PCIe back-end path and a set of memory modules;
a PCIe crossover switch having:
a front-end PCIe switch having a first plurality of PCIe lanes connected to one or more front-end PCIe host bus adaptors, said front-end host bus adaptors being connectable to one or more host paths, and a second plurality of PCIe lanes connected each of said PCIe front-end paths of said four PCIe memory controllers; and
a back-end PCIe switch having a first plurality of PCIE lanes connected to one or more back-end PCIe host bus adaptors, said back-end host bus adaptors being connectable to one or more drive paths, and a second plurality PCIe lanes connected to each of said PCIe back-end paths of said four PCIe memory controllers.
2. The device of claim 1 , further comprising a processor operative to schedule data into said memory modules of said four PCIe memory modules.
3. The device of claim 1 , wherein a total number of PCIe lanes connected to each of said PCIe front-end paths is at least equal to a total number of PCIe lanes connected to any of said front-end PCIe host bus adaptors, wherein said front-end PCIe switch provides full bandwidth from any host path to any PCIe memory controller.
4. The device of claim 1 , wherein any drive path of the back-end host bus adaptors is connectable to any host path of the front-end host bus adaptors via one of said four PCIe memory controllers.
5. The device of claim 1 , wherein said front-end PCIe switch comprises:
a fist set of first and second 96 lane PCIe switches, wherein each of said first and second 96 lane PCIe switches includes:
32 PCIe lanes connected to one or more of said front-end PCIe host bus adaptors;
16 PCIe lanes connected to first and second PCIe×16 front-end paths of a unique set of two of said four PCIe memory controllers; and
32 PCIE lanes connected to the other of the first set of first and second 96 lane PCIe switches.
6. The device of claim 5 , wherein said back-end PCIe switch comprises:
a second set of first and second 96 lane PCIe switches, wherein each of said first and second 96 lane PCIe switches includes:
32 PCIe lanes connected to one or more of said back-end PCIe host bus adaptors;
16 PCIe lanes connected to first and second PCIe×16 back-end paths of a unique set of two of said four PCIe memory controllers; and
32 PCIE lanes connected to the other of the second set of first and second 96 lane PCIe switches.
7. The device of claim 1 , wherein each said four PCIe memory controller is operative to store data to said set of memory modules with parity.
8. The device of claim 7 , wherein, each PCIe memory controller further comprises a data forking path, wherein data received by each said PCIe memory controller is copied to another PCIe memory controller.
9. The device of claim 1 , wherein memory of said memory modules of said four PCIe memory controllers is presented in a PCIe memory space.
10. The device of claim 9 , wherein said memory of said memory modules of said four PCIe memory controllers is further mapped into an address space of said processor.
11. The device of claim 1 , wherein at least one of said four PCIe memory controllers comprises field programmable gate array (FPGA).
12. The device of claim 11 , wherein said FPGA, further comprises:
a PCIe controller; and
PCIe registers.
13. The device of claim 1 , wherein at least one of said PCIe memory controller comprises:
a single socket CPU.
14. The device of claim 13 , wherein a processing core of at least one of at least one said single socket CPUs comprises the processor operative to schedule data into said memory modules.
15. The device of claim 13 , wherein said single socket CPUs comprise one of
a Sandy Bridge processing complex; and
an Ivy Bridge processing complex.
16. The device of claim 13 , wherein said four PCIe memory controllers is a first set of four PCIe memory controllers and said PCIe crossover switch is a first PCIe crossover switch, further comprising:
a second set of four PCIe memory controllers, each having a PCIe front-end path, a PCIe back-end path and a set of memory modules;
a second PCIe crossover switch having:
a second front-end PCIe switch having a first plurality of PCIe lanes connected to one or more front-end PCIe host bus adaptors, said front-end host bus adaptors being connectable to one or more host paths, and a second plurality of PCIe lanes connected each of said PCIe front-end paths of said second set of four PCIe memory controllers; and
a second back-end PCIe switch having a first plurality of PCIe lanes connected to one or more back-end PCIe host bus adaptors, said back-end host bus adaptors being connectable to one or more drive paths, and a second plurality of PCIe lanes connected to each of said PCIe back-end paths of said second set of four PCIe memory controllers.
17. The device of claim 16 , wherein, a data forking path of each of said first set of four PCIe memory controllers is connected to a data forking path of one of said second set of four PCIe memory controllers.