Granted Patent B2

US 12,248,424 · App. 18/447,180 · Granted Mar 11, 2025

Inventors: Thomas Norrie (Mountain View, CA); Frederic Vecoven (Mountain View, CA); Kiran Seshadri (Mountain View, CA); Shrijeet Mukherjee (Mountain View, CA); Chetan Loke (Mountain View, CA)

Assignee: Enfabrica Corporation

G06F13/4221G06F2213/0026

Abstract

Claims (69)

1. A system for sharing peripheral component interconnect express (PCIe) devices across multiple host servers comprising:

a plurality of hosts associated with a plurality of hierarchies;

one or more endpoints associated with one or more of the plurality of hierarchies; and

a switch communicatively connectable to the plurality of hosts and the one or more endpoints, wherein the switch is configured to:

receive a transaction layer packet (TLP);

determine a policy group identifier based on processing the TLP;

perform packet forward matching based on the policy group identifier and destination fields of the TLP;

based on whether the TLP is communicated between the hosts and endpoints in different hierarchies of the plurality of hierarchies, determine whether to edit the TLP using one or more rewrite rules; and

forward the TLP to an appropriate destination link.

2. The system of claim 1 , wherein the switch is further configured to determine the policy group identifier by compressing a source identifier of the TLP using a mapping function to generate the policy group identifier.

3. The system of claim 1 , wherein the destination fields include a destination identifier, and the matching is performed based on the destination identifier and the policy group identifier.

4. The system of claim 1 , wherein the destination fields include a destination address field, and the matching is performed based on the destination address field and the policy group identifier.

5. The system of claim 1 , wherein editing the TLP using the one or more rewrite rules further comprises one or more of rewriting a source identifier and tag of the TLP, rewriting a destination address or device identifier of the TLP, or rewriting a traffic class of the TLP.

6. The system of claim 5 , wherein the TLP is a non-posted request, and the switch is further configured to:

perform tag rewriting; and

track an original tag and source identifier of the non-posted request based on the tag rewriting.

7. The system of claim 1 , wherein the switch performs the packet forward matching using a ternary content addressable memory (TCAM) for long matches and fast lookups.

8. A system for sharing peripheral component interconnect express (PCIe) devices across multiple host servers such that a first host in a first hierarchy can write content to a memory range in a second hierarchy, the system comprising:

a plurality of hosts associated with a plurality of hierarchies, the plurality of hosts including the first host and a second host respectively associated with the first hierarchy and the second hierarchy in the plurality of hierarchies;

one or more endpoints associated with one or more of the plurality of hierarchies; and

a switch communicatively connectable to the plurality of hosts and the one or more endpoints, wherein the switch is configured to:

receive a first write packet from the first host to a first ghost device, wherein the first host and the first ghost device both reside in the first hierarchy;

perform packet forward matching on the first write packet;

rewrite and convert the first write packet into a second write packet based on the matching of the first write request, wherein the second write packet is sourced from the first ghost device in the first hierarchy and destinated to the second host in the second hierarchy; and

route the second write packet to the second host in the second hierarchy, wherein the second host is configured as a ghost endpoint server to communicate with an endpoint in the second hierarchy to write the content to the memory range in the second hierarchy.

9. The system of claim 8 , wherein ghost devices, including the first ghost device and the ghost endpoint server, are configured to provide device identifiers, memory space, and tag space for PCIe packets that bridge between the plurality of hierarchies.

10. The system of claim 9 , wherein the switch is further configured to:

map a ghost device in a hierarchy with a different ghost device in a different hierarchy, including mapping the first ghost device in the first hierarchy to the ghost endpoint server or second host in the second hierarchy through a ghost bridge, and

wherein the packet forward matching is performed based on mappings between the ghost devices.

11. The system of claim 10 , wherein the mapped ghost devices enable bridging ghost transactions between independent hierarchies of the plurality of hierarchies.

12. The system of claim 8 , wherein the first write request is converted into the second write packet using one or more rewrite rules, and the second write packet is routed to the ghost endpoint server that is matched for a specific rewrite rule.

13. The system of claim 8 , wherein the switch is configured to perform the packet forward matching on the first write packet by:

determining a policy group identifier associated with a source identifier of the first write packet; and

performing the packet forward matching based on the policy group identifier and destination fields of the first write packet.

14. The system of claim 8 , wherein the switch is further configured to:

rewrite a traffic class of the first write packet when converting the first write packet to the second write packet.

15. The system of claim 8 , wherein the switch is further configured to perform DMA ghost read and ghost completion based on address field rewriting.

16. A system for sharing a peripheral component interconnect express (PCIe) device across multiple host servers without involving a host managing the PCIe device, the system comprising:

a plurality of hosts associated with a plurality of hierarchies, the plurality of hosts including a first host associated with a first hierarchy in the plurality of hierarchies, wherein each host is connected to a ghost device, and each ghost device is composed of ghost physical functions and ghost virtual functions;

one or more endpoints associated with one or more of the plurality of hierarchies, the one or more endpoints including a second endpoint in a second hierarchy of the plurality of hierarchies; and

a switch communicatively connectable to the plurality of hosts and the one or more endpoints, wherein the switch is configured to:

receive a packet from the first host in the first hierarchy to a ghost virtual function of a first ghost device connected to the first host;

perform packet forward matching on the first packet to determine the ghost virtual function of the first ghost device is mapped to a virtual function of the second endpoint through a first ghost bridge;

convert the first packet into a second packet based on the matching, wherein the second packet is sourced from the first ghost bridge and destinated to the second endpoint in the second hierarchy; and

route the second packet to the second endpoint in the second hierarchy.

17. The system of claim 16 , wherein the first packet is a non-posted transaction layer packet, and to convert the first packet to the second packet, the switch is further configured to:

use tag rewriting to regenerate a completion source identifier and tag.

18. A method for sharing peripheral component interconnect express (PCIe) devices across multiple host servers comprising:

receiving a transaction layer packet (TLP);

determining a policy group identifier based on parsing and processing the TLP;

performing packet forward matching based on the policy group identifier and destination fields of the TLP;

based on whether the TLP is communicated between the hosts and endpoints in different hierarchies of the plurality of hierarchies, determining whether to edit the TLP using one or more rewrite rules; and

forwarding the TLP to an appropriate destination link.

19. The method of claim 18 , wherein the destination fields include a destination identifier, and the matching is performed based on the destination identifier and the policy group identifier.

20. The method of claim 18 , wherein the destination fields include a destination address field, and the matching is performed based on the destination address field and the policy group identifier.

21. A method for data access via PCIe between a plurality of hosts and a plurality of endpoints, the method comprising performing, at a ghost endpoint server, steps of:

designating for each host a respective control-plane ghost bridge for exchanging control information with any of the plurality of endpoints via the ghost endpoint server; and

designating for each host-endpoint pair a respective data-plane ghost bridge, wherein respective addresses of the data-plane ghost bridges are modified addresses, uniquely identifying for each host-endpoint pair, a destination of a data communication between the host-endpoint pair.

22. The method of claim 21 , wherein the respective destination of the data communication is an endpoint belonging to the respective host-endpoint pair, or a respective host belonging to the respective host-endpoint pair.

23. The method of claim 21 , further comprising:

associating with at least one host, a ghost control plane for exchanging control information with a corresponding control-plane ghost bridge.

24. The method of claim 21 , further comprising:

associating with at least one host, a plurality of ghost data planes, for exchanging data with respective data-plane ghost bridges.

25. A method for providing to a plurality of hosts access to a plurality of virtual functions (VFs) implemented at a plurality of endpoints, the method comprising:

designating a respective ghost bridge for each host, wherein each ghost bridge:

exposes one or more VFs from one or more of the plurality of endpoints as a set of ghost virtual functions (GVFs) to the corresponding host, and

maps each GVF to a corresponding VF; and

translating and forwarding a request from a host to execute a particular GVF exposed by the corresponding ghost bridge to a corresponding VF at one of the plurality of endpoints.

26. The method of claim 25 , wherein a ghost bridge corresponding to a particular host exposes only a subset of VFs from one of the plurality of endpoints.

Assignments (1)

Continuity (2)

References Cited (127)