Port-based routing (PBR) switches, compute express link (CXL) fabric, and CXL switch to manage cache coherency between host servers
A compute express link (CXL) fabric configured to connect a plurality of host servers and a plurality of devices to form a CXL network includes: one or more port-based routing (PBR) switches configured to connect the host servers, the devices, or other PBR switches to implement the CXL network; and a fabric manager configured to collectively manage the one or more PBR switches.
1 . A compute express link (CXL) fabric that connects at least one host server and a plurality of devices to form a single CXL network, the CXL fabric comprising:
at least one port-based routing (PBR) switch configured to be connected to the at least one host server, at least one of the plurality of devices, or another PBR switch to implement the CXL network; and
a fabric manager configured to manage the at least one PBR switch,
wherein the at least one PBR switch is configured to receive information about devices connected to each port,
wherein the at least one PBR switch is configured to:
discriminate a port of a CXL 2.0 host and a downstream port of a hierarchy-based routing (HBR) switch, based on modified TS (training sequence) 1/TS2 ordered sets transmitted and received in a peripheral component interconnect-express (PCIe) link-based alternate protocol negotiation (APN) process with a neighboring port.
2 . A compute express link (CXL) fabric that connects at least one host server and a plurality of devices to form a single CXL network, the CXL fabric comprising:
at least one port-based routing (PBR) switch configured to be connected to the at least one host server, at least one of the plurality of devices, or another PBR switch to implement the CXL network; and
a fabric manager configured to manage the at least one PBR switch,
wherein the at least one PBR switch is configured to receive information about devices connected to each port,
wherein the fabric manager is configured to determine a topology of the CXL network based on information about the devices,
wherein the at least one PBR switch is configured to discriminate a PBR switch and a global fabric attached memory (G-FAM) device (GFD) based on a vendor-defined message (VDM).
3 . A compute express link (CXL) fabric that connects at least one host server and a plurality of devices to form a single CXL network, the CXL fabric comprising:
at least one port-based routing (PBR) switch configured to be connected to the at least one host server, at least one of the plurality of devices, or another PBR switch to implement the CXL network; and
a fabric manager configured to manage the at least one PBR switch,
wherein the at least one PBR switch is configured to receive information about devices connected to each port,
wherein the fabric manager is configured to determine a topology of the CXL network based on information about the devices,
wherein the fabric manager is configured to:
assign PBR identifiers (IDs) for PBR flit routing to the upstream port and the downstream port of the at least one PBR switch; and
generate connectivity based on the PBR IDs to determine the topology of the CXL network.
4 . The CXL fabric of claim 3 , wherein the connectivity is represented by an adjacency matrix that treats a PBR switch as a node and represents the connectivity between the PBR switches.
5 . A compute express link (CXL) fabric that connects at least one host server and a plurality of devices to form a single CXL network, the CXL fabric comprising:
at least one port-based routing (PBR) switch configured to be connected to the at least one host server, at least one of the plurality of devices, or another PBR switch to implement the CXL network; and
a fabric manager configured to manage the at least one PBR switch,
wherein the at least one PBR switch is configured to store metadata for identifying the at least one PBR switch as at least some virtual switch in response to a device enumeration process of the host server.
6 . The CXL fabric of claim 5 , wherein the virtual switch corresponds to one host server, and is configured to directly connect the host server and at least one device assigned to the host server.
7 . A compute express link (CXL) fabric that connects at least one host server and a plurality of devices to form a single CXL network, the CXL fabric comprising:
at least one port-based routing (PBR) switch configured to be connected to the at least one host server, at least one of the plurality of devices, or another PBR switch to implement the CXL network; and
a fabric manager configured to manage the at least one PBR switch,
wherein the at least one PBR switch is configured to perform a conversion between a hierarchy-based routing (HBR) message and a PBR message based on prestored metadata.
8 . The CXL fabric of claim 7 , wherein the conversion between the HBR message and the PBR message is performed on ports comprised in the PBR switch,
wherein a port connected on a message generation side is configured to perform a message format conversion by including a destination PBR ID (DPID) and a source PBR ID (SPID) in an incoming message into the port, and
a port connected on a message end side is configured to perform a message format conversion by excluding the DPID and the SPID from an incoming message into the port.
9 . The CXL fabric of claim 8 , wherein the SPID comprises a PBR ID of the port connected on the message generation side or an ID value comprised in the incoming message, and
the DPID comprises a PBR ID of the port connected on the message end side.
10 . A port-based routing (PBR) switch, comprising:
a routing table configured to store routing information associated with routing paths in a compute express link (CXL) network;
two or more ports configured to function as an upstream port, a downstream port, or a fabric port;
a crossbar switch configured to set a connection path (crossbar) between the two or more ports based on the routing information; and
a controller configured to perform monitoring and setting changes on the routing table, the two or more ports, and the crossbar switch,
wherein each of the two or more ports comprises:
a multiplexer configured to determine a transmission path within a port for an incoming message into the port;
a format conversion module configured to convert a format of the incoming message; and
an edge port controller configured to perform initialization and setting on an edge port.
11 . A port-based routing (PBR) switch, comprising:
a routing table configured to store routing information associated with routing paths in a compute express link (CXL) network;
two or more ports configured to function as an upstream port, a downstream port, or a fabric port;
a crossbar switch configured to set a connection path (crossbar) between the two or more ports based on the routing information; and
a controller configured to perform monitoring and setting changes on the routing table, the two or more ports, and the crossbar switch,
wherein each of the two or more ports is configured to transmit only a PBR message to the crossbar switch regardless of a format of an incoming message into a port.
12 . A compute express link (CXL) switch, comprising:
a port;
a snoop filter connected to the port and configured to perform cache coherency management between host servers sharing a specific device; and
a static random-access memory (SRAM) configured to store states and owner information of cache line data used by the snoop filter for the cache coherency management;
wherein, in response to the CXL switch being a port-based routing (PBR) switch, a size of the owner information is calculated based on the a number of all ports comprised in the PBR switch, and
in response to the CXL switch being a hierarchy-based routing (HBR) switch, the size of the owner information is calculated based on the a number of upstream ports comprised in the HBR switch.
13 . The CXL switch of claim 12 , wherein a size of the owner information is calculated based on a number of ports, not on a number of host servers.
14 . The CXL switch of claim 12 , wherein the snoop filter is configured to:
perform the cache coherency management between the host servers, based on a source PBR identifier (ID) (SPID), a destination PBR ID (DPID), or a logical device ID (LD-ID) comprised in a message reaching itself.
15 . The CXL switch of claim 12 , wherein a set of CXL switches constitutes a back-invalidation-based cache-coherent network,
wherein only CXL switches that do not have the same port used when transmitting a snoop request to a host server among the host servers participate in the cache coherency management between the host servers.
16 . The CXL switch of claim 12 , wherein a number of cache line data stored in the SRAM is calculated based on the number of host servers for which the CXL switch manages cache coherency and a total sum of remaining capacities of all SRAMs comprised in network, the all SRAMs including the SRAM.
17 . The CXL switch of claim 12 , wherein, of the port, an upstream port comprises:
a protocol conversion module configured to perform a protocol conversion based on a mapping table between a host-managed device memory-D (HDM device coherent, HDM-D) and a host-managed device memory-DB (HDM device coherent using back-invalidation, HDM-DB), for cache coherency management of host servers that do not support a back-invalidation operation.