Input/output system interconnect redundancy and failover
A system and method for achieving peripheral component interconnect express (PCIe) redundancy and recovery are disclosed. In some embodiments, the system comprises an accelerated compute fabric (ACF) comprising a PCIe switch, an application host communicatively coupled to the ACF using one or more upstream PCIe links, and an endpoint device communicatively coupled to the ACF using one or more downstream PCIe links. The application host is configured to send PCIe transaction layer packets (TLP) addressed to the ghosted endpoint devices through the one or more upstream PCIe links, and the ACF is configured to redirect the PCIe TLP packets to the endpoint device through the one or more downstream PCIe links.
1. A system for achieving peripheral component interconnect express (PCIe) redundancy and recovery, the system comprising:
an accelerated compute fabric (ACF) comprising a PCIe switch;
an application host communicatively coupled to the ACF using one or more upstream PCIe links; and
an endpoint device communicatively coupled to the ACF using one or more downstream PCIe links,
wherein the application host is configured to send PCIe transaction layer packets (TLP) addressed to a plurality of ghosted endpoint devices through the one or more upstream PCIe links,
wherein the ACF is configured to redirect the PCIe TLP packets to the endpoint device through the one or more downstream PCIe links, and
wherein a total number of ghosted endpoint devices in the plurality of ghosted endpoint devices is equal to a product of (i) a number of links in the one or more upstream PCIe links and (ii) a number of links in the one or more downstream PCIe links.
2. The system of claim 1 , wherein the ACF comprises one or more PCIe ports, and wherein the ACF uses the PCIe switch and the one or more PCIe ports to transfer data between the application host and the endpoint device.
3. The system of claim 1 , wherein the application host comprises a computing device configured to perform operations for at least one of machine learning or artificial intelligence.
4. The system of claim 1 , wherein the endpoint device comprises at least one of a graphics processing unit, a storage or memory element, a hard disk drive, a network interface card, or a field-programmable gate array.
5. The system of claim 1 , wherein, to send a block of data to the endpoint device, the application host is configured to distribute portions of the block of data across the PCIe TLP packets, with each PCIe TLP packet addressed to a respective ghosted endpoint device from the plurality of ghosted endpoint devices.
6. The system of claim 1 , wherein each upstream PCIe link from the one or more upstream PCIe links is associated with one or more respective ghosted endpoint devices from the plurality of ghosted endpoint devices.
7. The system of claim 1 , wherein the application host is communicatively coupled to the ACF using a plurality of upstream PCIe links and, when none of the upstream PCIe links from the plurality of upstream PCIe links is in a failed state, the application host is configured to send the PCIe TLP packets through all the upstream PCIe links of the plurality of upstream PCIe links.
8. The system of claim 7 , wherein, to send the PCIe TLP packets through all the upstream PCIe links from the plurality of upstream PCIe links, the application host is configured to address at least one of the PCIe TLP packets to each ghosted endpoint devices from the plurality of ghosted endpoint devices.
9. The system of claim 1 , wherein the application host is communicatively coupled to the ACF using a plurality of upstream PCIe links and, when an upstream PCIe link from the plurality of upstream PCIe links is in a failed state, the application host is configured to send the PCIe TLP packets through one or more remaining upstream PCIe links of the plurality of upstream PCIe links.
10. The system of claim 9 , wherein, to send the PCIe TLP packets through the one or more remaining upstream PCIe links, the application host is configured to address the PCIe TLP packets to one or more ghosted endpoint devices from the plurality of ghosted endpoint devices that are associated with the one or more remaining upstream PCIe links.
11. The system of claim 1 , wherein the endpoint device is communicatively coupled to the ACF using a plurality of downstream PCIe links and, when none of the downstream PCIe links from the plurality of downstream PCIe links is in a failed state, the ACF is configured to redirect the PCIe TLP packets to the endpoint device through all the downstream PCIe links of the plurality of downstream PCIe links.
12. The system of claim 1 , wherein the endpoint device is communicatively coupled to the ACF using a plurality of downstream PCIe links and, when a downstream PCIe link from the plurality of downstream PCIe links is in a failed state, the ACF is configured to redirect the PCIe TLP packets to the endpoint device through one or more remaining downstream PCIe links of the plurality of downstream PCIe links.
13. The system of claim 1 , wherein each ghosted endpoint device from the plurality of ghosted endpoint devices is a ghost copy of the endpoint device.
14. The system of claim 1 , further comprising a controlling host communicatively coupled to the ACF and configured to send data to the endpoint device.
15. A method of achieving peripheral component interconnect express (PCIe) redundancy and recovery, the method comprising:
providing an accelerated compute fabric (ACF) comprising a PCIe switch, wherein the ACF is communicatively coupled using one or more upstream PCIe links to an application host, and wherein the ACF is communicatively coupled using one or more downstream PCIe links to an endpoint device,
receiving, by the ACF, PCIe TLP packets sent by the application host through the one or more upstream PCIe links and addressed to a plurality of ghosted endpoint devices; and
redirecting, by the ACF, the PCIe TLP packets to the endpoint device through the one or more downstream PCIe links,
wherein a total number of ghosted endpoint devices in the plurality of ghosted endpoint devices is equal to a product of (i) a number of links in the one or more upstream PCIe links and (ii) a number of links in the one or more downstream PCIe links.
16. The method of claim 15 , wherein receiving the PCIe TLP packets comprises receiving portions of a block of data across the PCIe TLP packets, with each PCIe TLP packet addressed to a respective ghosted endpoint device from the plurality of ghosted endpoint devices.
17. The method of claim 15 , wherein the ACF is communicatively coupled to the application host using a plurality of upstream PCIe links, and, when none of the upstream PCIe links from the plurality of upstream PCIe links is in a failed state, the PCIe TLP packets are received by the ACF through all the upstream PCIe links of the plurality of upstream PCIe links.
18. The method of claim 15 , wherein the ACF is communicatively coupled to the application host using a plurality of upstream PCIe links, and, when an upstream PCIe link from the plurality of upstream PCIe links is in a failed state, the PCIe TLP packets are received by the ACF through one or more remaining upstream PCIe links of the plurality of upstream PCIe links.
19. The method of claim 15 , wherein the ACF is communicatively coupled to the endpoint device using a plurality of downstream PCIe links, and, when none of the downstream PCIe links from the plurality of downstream PCIe links is in a failed state, the PCIe TLP packets are redirected, by the ACF, to the endpoint device through all the downstream PCIe links of the plurality of downstream PCIe links.
20. The method of claim 15 , wherein the ACF is communicatively coupled to the endpoint device using a plurality of downstream PCIe links, and, when a downstream PCIe link from the plurality of downstream PCIe links is in a failed state, the PCIe TLP packets are redirected, by the ACF, to the endpoint device through one or more remaining downstream PCIe links of the plurality of downstream PCIe links.