Composable infrastructure enabled by heterogeneous architecture, delivered by CXL based cached switch SoC
Described herein are systems, methods, and products utilizing a cache coherent switch on chip. The cache coherent switch on chip may utilize Compute Express Link (CXL) interconnect open standard and allow for multi-host access and the sharing of resources. The cache coherent switch on chip provides for resource sharing between components while independent of a system processor, removing the system processor as a bottleneck. Cache coherent switch on chip may further allow for cache coherency between various different components. Thus, for example, memories, accelerators, and/or other components within the disclose systems may each maintain caches, and the systems and techniques described herein allow for cache coherency between the different components of the system with minimal latency.
1 . A system comprising:
a first cache coherent switch on chip in communication with a first memory device via a Compute Express Link (CXL); and
a second cache coherent switch on chip in communication with a second memory device and the first cache coherent switch on chip via the CXL;
wherein the first cache coherent switch on chip and the second cache coherent switch on chip are configured to:
provide a memory pool from the first memory device and the second memory device that bypasses a central processing unit (CPU); and
maintain, via the memory pool, cache coherency between a plurality of components of different component types.
2 . The system of claim 1 , wherein the first memory device and the second memory device are on a same device or a different device.
3 . The system of claim 1 , wherein the first memory device is a different component type than the second memory device.
4 . The system of claim 1 , wherein the first memory device is random access memory and the second memory device is persistent memory, and wherein the first memory device is configured to provide caching for the second memory device.
5 . The system of claim 1 , wherein the first cache coherent switch on chip is further configured to determine that data is not stored in the first memory device and obtain the data from the second memory device via the second cache coherent switch on chip.
6 . A method comprising:
configuring, by a fabric manager, a first cache coherent switch on chip to be in communication with a first memory device via a Compute Express Link (CXL);
configuring, by the fabric manager, a second cache coherent switch on chip to be in communication with a second memory device and the first cache coherent switch on chip via the CXL;
using, by the fabric manager, the cache coherent switch on chip and the second cache coherent switch on chip to create a memory pool from the first memory device and the second memory device that bypasses a central processing unit (CPU); and
maintaining, via the memory pool, cache coherency between a plurality of components of different component types.
7 . The method of claim 6 , wherein the first memory device and the second memory device are on a same device.
8 . The method of claim 6 , wherein the first memory device and the second memory device are in different devices.
9 . The method of claim 6 , wherein the memory pool is cache coherent.
10 . The method of claim 6 , wherein the first memory device is a different type of memory device than the second memory device.
11 . The method of claim 6 , wherein the first memory device is random access memory and the second memory device is persistent memory, and wherein the first memory device is configured to provide caching for the second memory device.
12 . The method of claim 6 , further comprising determining, by the first cache coherent switch on chip, that data is not stored in the first memory device and obtaining the data from the second memory device via the second cache coherent switch on chip.
13 . A system comprising:
a first cache coherent switch on chip, in communication with a first memory device via a Compute Express Link (CXL), and a second cache coherent switch on chip, in communication with a second memory device via the CXL, are configured to pool memory from the first memory device and the second memory device, wherein the first memory device and the second memory device bypass a central process unit (CPU) and the pooled memory is used to maintain cache coherency between a plurality of components of different component types; and
a memory prefetcher of the first cache coherent switch on chip configured to prefetch data from the first memory device or the second memory device to store in one or more buffers.
14 . The system of claim 13 , wherein the first cache coherent switch on chip is further configured to determine the data to prefetch.
15 . The system of claim 13 , wherein the memory prefetcher of the first cache coherent switch on chip is further configured to request data from the second cache coherent switch on chip.
16 . The system of claim 15 , wherein the memory prefetcher of the first cache coherent switch on chip is further configured to request data from the second cache coherent switch on chip responsive to determining the data is not stored within the first memory device.
17 . The system of claim 13 , wherein the first cache coherent switch on chip is on a first device and the second cache coherent switch on chip is on a second device.
18 . The system of claim 13 , wherein the first memory device and the second memory device are on different devices.
19 . The system of claim 1 , wherein the different component types include at least one of volatile memory, persistent memory, solid state drive, input/output (I/O) device, artificial intelligence accelerator, graphics processing unit (GPU), hardware accelerator, and field programmable gate array (FPGA).
20 . The system of claim 1 , wherein the first cache coherent switch on chip and the second cache coherent switch on chip are further configured to:
prioritize reading or writing data from the memory pool associated with a first component type of the different component types over reading or writing data associated with a second component type of the different component types.