Adaptive fabric allocation for local and remote emerging memories based prediction schemes
Methods, apparatus and systems for adaptive fabric allocation for local and remote emerging memories-based prediction schemes. In conjunction with performing memory transfers between a compute host and memory device connected via one or more interconnect segments, memory read and write traffic is monitored for at least one interconnect segment having reconfigurable upstream lanes and downstream lanes. Predictions of expected read and write bandwidths for the at least one interconnect segment are then made. Based on the expected read and write bandwidths, the upstream lanes and downstream lanes are dynamically reconfigured. The interconnect segments include interconnect links such as Compute Exchange Link (CXL) flex buses and memory channels for local memory implementations, and fabric links for remote memory implementations. For local memory, management messages may be used to provide telemetry information containing the expected read and write bandwidths. For remote memory, telemetry information is provided to a fabric management component that is used to dynamically reconfigure one or more fabric links.
1. A method comprising:
coupling a compute host to a memory device via an interconnect structure including one or more interconnect segments with a plurality of reconfigurable upstream lanes and downstream lanes;
performing memory reads and writes initiated by the host to read data from the memory device and write data to the memory device via the interconnect structure;
for each of the at least one of the one or more interconnect segments with the plurality of reconfigurable upstream lanes and downstream lanes,
monitoring memory read and memory write traffic transferred via that interconnect segment;
predicting, based on the monitored memory read and memory write traffic, expected read and write bandwidths for that interconnect segment; and
dynamically reconfiguring the plurality of upstream lanes and downstream lanes for that interconnect segment based on the expected read and write bandwidths for that interconnect segment;
wherein at least one interconnect segment for which the plurality of upstream and downstream lanes is dynamically reconfigured comprises a Compute Express Link (CXL) flex bus, and wherein the memory device is installed in a CXL device that performs the monitoring and prediction operations.
2. The method of claim 1 , wherein the memory device comprises a Dual In-line Memory Module (DIMM).
3. The method of claim 1 , further comprising:
sending expected read and write bandwidth usage data for each of at least one interconnect segment for which expected read and write bandwidth usage is predicted to a bandwidth management component; and
utilizing the bandwidth management component to dynamically reconfigure the plurality of upstream and downstream lanes for that interconnect segment based on the expected read and write bandwidth usage data.
4. A Compute Exchange Link (CXL) card, comprising:
a CXL interface configured to be connected to a CXL flex bus via which a compute host is directly or indirectly connected, the CXL flex bus including a plurality of reconfigurable upstream lanes and downstream lanes; and
a Dual In-line Memory Module (DIMM) coupled to the CXL interface via an internal interconnect having logic configured to,
monitor memory read and memory write traffic transferred via the internal interconnect to and from the memory device; and
predict, based on the memory read and memory write traffic that is monitored, expected read and write bandwidths for the internal interconnect; or,
if the memory device includes the external link interface, monitor memory read and memory write traffic transferred via the external interconnect to and from the memory device; and
predict, based on the memory read and memory write traffic that is monitored, expected read and write bandwidths for the external link.
5. The CXL card of claim 4 , further comprising logic to:
send telemetry information comprising the expected read and write bandwidths predicted for the external link to the compute host;
receive lane configuration information from the compute host; and dynamically reconfigure the plurality of upstream lanes and downstream lanes for the external link based on the lane configuration information received from the compute host.
6. A system comprising:
a processor having a System on a Chip (SoC) architecture comprising, a plurality of processor cores, each including one or more cache levels and coupled to an interconnect;
at least one memory controller having one or more memory channels including a first memory channel, coupled to the interconnect; and
a Compute Exchange Link (CXL) interface, coupled to the interconnect;
a first Dual In-line Memory Module (DIMM), coupled to the first memory channel; and
one of a CXL device to which a second DIMM is coupled or installed or a CXL DIMM, coupled to the CXL interface via a CXL flex bus having a plurality of upstream lanes and downstream lanes, wherein the system is configured to,
monitor memory read and memory write traffic transferred via the CXL flex bus to the second DIMM or CXL DIMM;
predict, based on the monitored memory read and memory write traffic, expected read and write bandwidths for the CXL flex bus; and
dynamically reconfigure the plurality of upstream lanes and downstream lanes for the CXL flex bus based on the expected read and write bandwidths that are predicted.
7. The system of claim 6 , wherein the second DIMM or CXL DIMM includes logic configured to implement the monitoring and predicting operations.
8. The system of claim 6 , wherein the second DINM is installed in or coupled to a CXL device including logic configured to implement the monitoring and predicting operations.
9. The apparatus of claim 6 , wherein the processor further includes a dynamic bandwidth allocation block; and
wherein the system is further configured to:
send telemetry information comprising the expected read and write bandwidths for the CXL flex bus to the processor; and
determine, using the dynamic bandwidth allocation block, a lane configuration to be used for the CXL flex bus; and
dynamically reconfigure the plurality of upstream lanes and downstream lanes for the CXL flex bus based on the lane configuration determined by the dynamic bandwidth allocation block.
10. The apparatus of claim 6 , further comprising:
sending the telemetry information from the CXL device or CXL DIMM over the CXL flex bus to the processor using one or more first CXL management messages;
returning lane configuration information corresponding to the lane configuration determined by the dynamic bandwidth allocation block from the processor to the CXL device or CXL DIMM over the CXL flex bus using one or more second CXL management messages; and
processing the lane configuration information at the CXL device or CXL DIMM to dynamically reconfigure the plurality of upstream lanes and downstream lanes for the CXL flex bus.