Composable infrastructure enabled by heterogeneous architecture, delivered by CXL based cached switch SoC
Described herein are systems, methods, and products utilizing a cache coherent switch on chip. The cache coherent switch on chip may utilize Compute Express Link (CXL) interconnect open standard and allow for multi-host access and the sharing of resources. The cache coherent switch on chip provides for resource sharing between components while independent of a system processor, removing the system processor as a bottleneck. Cache coherent switch on chip may further allow for cache coherency between various different components. Thus, for example, memories, accelerators, and/or other components within the disclose systems may each maintain caches, and the systems and techniques described herein allow for cache coherency between the different components of the system with minimal latency.
1. A cache coherent switch on chip comprising:
a plurality of ports, each port configured to communicatively couple to an associated component of a plurality of components and configured to support a Compute Express Link (CXL) protocol;
switched fabric circuitry, communicatively coupled to the plurality of ports and comprising:
switched fabric circuitry configured in a mesh network comprising a plurality of nodes; and
a microprocessor, communicatively coupled to the switched fabric circuitry and comprising:
a fabric manager, configured to allocate and/or deallocate resources to one or more of the plurality of ports and;
wherein the cache coherent switch on chip is configured to share resources between the plurality of components utilizing the CXL protocol over a CXL interface while bypassing a central processing unit to share the resources.
2. The cache coherent switch on chip of claim 1 , wherein the cache coherent switch on chip further comprises:
a plurality of virtual hierarchies, wherein the fabric manager is configured to:
determine that a first component has been coupled to a first port; and
assign the first component to a first virtual hierarchy.
3. The cache coherent switch on chip of claim 2 , wherein the first port comprises a plurality of virtual PCI-to-PCI bridges (vPPB) and a first PCI-to-PCI bridge (PPB), wherein the first component is associated with a PPB, and wherein the fabric manager is further configured to:
determine an attribute associated with the first component, wherein the assigning the first component to the first virtual hierarchy is based on the attribute;
assign the first component to a first vPPB based on the attribute; and
bind the first vPPB to the first PPB.
4. The cache coherent switch on chip of claim 1 , wherein the plurality of ports comprises an upstream port, a first downstream port, and a second downstream port, wherein the fabric manager is further configured to:
receive a signal from the upstream port; and
direct the signal to the first downstream port.
5. The cache coherent switch on chip of claim 1 , wherein the fabric manager is implemented within firmware of the microprocessor.
6. The cache coherent switch on chip of claim 1 , wherein the processor is configured to:
detect, via the switched fabric circuitry, that a component has been hot-inserted; and
assign, with the fabric manager, the hot-inserted component to a first port of the plurality of ports.
7. The cache coherent switch on chip of claim 6 , wherein the first component is associated with a PCI-to-PCI bridge (PPB), and wherein the assigning the hot-inserted component to the first port comprises:
assigning the hot-inserted component to a first virtual hierarchy; and
binding a PPB to a first virtual PCI-to-PCI bridge (vPPB).
8. The cache coherent switch on chip of claim 6 , wherein the processor is further configured to:
perform diagnostics on the component, wherein the assigning the component to the first port is based on the diagnostics.
9. The cache coherent switch on chip of claim 1 , wherein the plurality of ports is communicatively coupled to a plurality of devices, each of the plurality of devices comprising a device cache, and wherein the microprocessor is configured to:
determine, based on the plurality of devices, a cache priority for the plurality of devices; and
manipulate the device caches according to the cache priority.
10. The cache coherent switch on chip of claim 9 , wherein the manipulating the device caches comprises one or more of fetching, reading, or writing cache data.
11. The cache coherent switch on chip of claim 9 , wherein the microprocessor is further configured to:
cache first data with at least one of the device caches;
receive a request for the first data, the request comprising tag RAM;
perform, through the tag RAM, a look-up for the first data of all of the plurality of device caches;
determine, based on the look-up, the cache location of the first data within the plurality of device caches;
access the first data; and
provide the first data.
12. The cache coherent switch on chip of claim 1 , further comprising:
a machine learning memory prefetcher, configured to predict addresses of access to memory coupled to one or more of the plurality of ports and prefetch data from the memory, wherein the prediction is based on an identity of applications communicatively coupled to the one or more of the plurality of ports.
13. The cache coherent switch on chip of claim 1 , wherein the fabric manager is configured to communicate with a management software entity and receive instructions from the management software entity.
14. The cache coherent switch on chip of claim 1 , wherein the associated component is one of a memory, an accelerator, an application specific integrated circuit, and/or storage.
15. The cache coherent switch on chip of claim 1 , further comprising:
the associated component.
16. A system comprising:
a first server device comprising:
a first memory device; and
a first cache coherent switch on chip, communicatively coupled to the first memory device, wherein the first cache coherent switch on chip comprises:
a plurality of ports, each port configured to communicatively couple to an associated component of a plurality of components and configured to support a Compute Express Link (CXL) protocol;
switched fabric circuitry, communicatively coupled to the plurality of ports and comprising:
switched fabric circuitry configured in a mesh network comprising a plurality of nodes; and
a microprocessor, communicatively coupled to the switched fabric circuitry and comprising:
a fabric manager, configured to allocate and/or deallocate resources to one or more of the plurality of ports;
wherein the cache coherent switch on chip is configured to share resources between the plurality of components utilizing the CXL protocol over a CXL interface while bypassing a central processing unit to share the resources.
17. The system of claim 16 , wherein the first cache coherent switch on chip further comprises a Random Access Memory (RAM) module, and wherein the system further comprises:
a DDR module communicatively coupled to the first cache coherent switch on chip via at least a first port of the plurality of ports;
persistent memory communicatively coupled to the first cache coherent switch on chip via at least a second port of the plurality of ports; and
a solid state drive communicatively coupled to the first cache coherent switch on chip via at least a third port of the plurality of ports.
18. The system of claim 17 , wherein the DDR module is external memory communicatively coupled via switch fabric.
19. The system of claim 17 , wherein the DDR module comprises DDR4 or DDR5 Synchronous Dynamic Random Access Memory (SDRAM).
20. The system of claim 16 , further comprising:
a second server device, communicatively coupled to the first server device via a data connection, the second server device comprising:
a second memory device; and
a second cache coherent switch on chip.