Arbitration across shared memory pools of disaggregated memory devices
Technology for a memory pool arbitration apparatus is described. The apparatus can include a memory pool controller (MPC) communicatively coupled between a shared memory pool of disaggregated memory devices and a plurality of compute resources. The MPC can receive a plurality of data requests from the plurality of compute resources. The MPC can assign each compute resource to one of a set of compute resource priorities. The MPC can send memory access commands to the shared memory pool to perform each data request prioritized according to the set of compute resource priorities. The apparatus can include a priority arbitration unit (PAU) communicatively coupled to the MPC. The PAU can arbitrate the plurality of data requests as a function of the corresponding compute resource priorities.
1. A memory pool arbitration apparatus, comprising:
a memory pool controller (MPC) configured to communicatively couple between a shared memory pool of disaggregated memory devices and a plurality of compute resources, the MPC to:
receive a plurality of data requests from the plurality of compute resources;
assign each compute resource to one of a set of compute resource priorities;
assign a bandwidth allocation to each compute resource according to a corresponding compute resource priority; and
send memory access commands to the shared memory pool to perform each data request prioritized according to the set of compute resource priorities;
a priority arbitration unit (PAU) communicatively coupled to the MPC, the PAU to arbitrate the plurality of data requests as a function of the corresponding compute resource priorities; and
wherein the MPC further comprises a resource monitor unit to:
monitor a bandwidth usage for each compute resource; and
send the bandwidth usage for each compute resource to the PAU, wherein the PAU is further to:
determine an available bandwidth of the shared memory pool; and
if the available bandwidth is below an available bandwidth threshold, regulate the bandwidth allocations of at least a portion of the plurality of compute resources to maximize an overall bandwidth usage of the shared memory pool as a weighted function of the set of compute resource priorities.
2. The apparatus of claim 1 , wherein the PAU is further to:
generate a queue of the plurality of data requests for each compute resource priority; and
generate a data request sequence by arbitrating the data requests in the queues as a weighted function of the plurality of compute resource priorities.
3. The apparatus of claim 2 , wherein the MPC is further to: regulate bandwidth of data requests from a compute resource to modify a frequency of data requests from the compute resource in the data request sequence.
4. The apparatus of claim 2 , wherein the PAU is further to: regulate entry of data requests from a compute resource into the queue for the compute resource priority of the compute resource to modify a frequency of data requests from the compute resource in the data request sequence.
5. The apparatus of claim 2 , wherein:
the MPC is further to assign a priority indicator to each of the plurality of data requests based on the compute resource priority of the corresponding compute resource; and
the PAU is further to generate the data request sequence as a function of the priority indicators of the plurality of data requests weighted according to the corresponding compute resource priorities.
6. The apparatus of claim 1 , wherein, to regulate the bandwidth allocations of at least a portion of the plurality of compute resources, the PAU is further to:
compare the bandwidth usage to the bandwidth allocation for each of the plurality of compute resources;
identify a usage inefficiency of the shared memory pool from the comparison of the bandwidth usages to the bandwidth allocations; and
regulate the bandwidth allocations of at least a portion of the plurality of compute resources to reduce the usage inefficiency according to the weighted function of the set of compute resource priorities.
7. The apparatus of claim 6 , wherein the usage inefficiency comprises unused bandwidth of the shared memory pool, and the PAU is further configured to notify the MPC to ignore the bandwidth allocation of at least a portion of the plurality of compute resources to utilize the unused bandwidth of the shared memory pool.
8. The apparatus of claim 6 , wherein the PAU is further to regulate the bandwidth allocations by enforcing bandwidth limits, wherein the bandwidth limits are established by a service level agreement (SLA).
9. The apparatus of claim 1 , wherein at least a portion of the plurality of compute resources are each a compute node.
10. The apparatus of claim 1 , wherein at least a portion of the plurality of compute resources are each a workload.
11. The apparatus of claim 10 , wherein a compute resource can include multiple compute nodes executing the same workload.
12. The apparatus of claim 11 , wherein the MPC is further to assign a resource monitor identification (ID) to each of the plurality of compute resources, wherein the multiple compute nodes executing the same workload are monitored by the resource monitor according to the resource monitor ID.
13. The apparatus of claim 1 , wherein at least a portion of the plurality of compute resources are each a virtual machine.
14. The apparatus of claim 1 , wherein the MPC further comprises:
a controller to assign a resource monitor identification (ID) to each of the plurality of compute resources;
a bandwidth regulator to control a level of data bandwidth between the plurality of compute resources and the shared memory pool; and
a resource monitor unit to:
monitor a data bandwidth usage for each resource monitor ID; and
send the data bandwidth usage for each resource monitor ID to the bandwidth regulator.
15. A data center system, comprising:
a shared memory pool;
a plurality of compute resources communicatively coupled to the shared memory pool;
a memory pool controller (MPC) communicatively coupled between the shared memory pool and the plurality of compute resources to control data access, the MPC to:
assign a bandwidth allocation to each compute resource according to a corresponding compute resource priority;
receive a plurality data requests from the plurality of compute resources;
assign a priority indicator to each of the plurality of data requests; and
send memory access commands to the shared memory pool to fill each data request according to the priority indicator; and
a priority arbitration unit (PAU) communicatively coupled to the MPC, the PAU to generate the priority indicator for each data request based on a compute resource priority assigned to each compute resource; and
wherein the MPC further comprises a resource monitor unit to:
monitor a bandwidth usage for each compute resource; and
send the bandwidth usage for each compute resource to the PAU, wherein the PAU is further to:
determine an available bandwidth of the shared memory pool; and
if the available bandwidth is below an available bandwidth threshold, regulate the bandwidth allocations of at least a portion of the plurality of compute resources to maximize an overall bandwidth usage of the shared memory pool as a weighted function of the corresponding compute resource priority.
16. The system of claim 15 , wherein the PAU is further to: distribute the plurality of data requests to the shared memory pool based on the compute resource priority according to an arbitration schedule.
17. The system of claim 15 , wherein the compute resource priority includes at least two priority levels, and the PAU distributes the plurality of data requests to the shared memory pool as a function of an available shared memory pool resource.
18. The system of claim 17 , wherein the shared memory pool resource comprises memory bandwidth, memory access latency, memory speed, or a combination thereof.
19. A method for memory pool arbitration, comprising:
receiving, at a memory pool controller (MPC) communicatively coupled between a shared memory pool of disaggregated memory devices and a plurality of compute resources in a data center system, a plurality of data requests from the plurality of compute resources;
assigning each compute resource to one of a set of compute resource priorities;
sending memory access commands to the shared memory pool to perform each data request prioritized according to the set of compute resource priorities, wherein the plurality of data requests are arbitrated as a function of corresponding compute resource priorities;
determining an available bandwidth of the shared memory pool; and
regulating, when the available bandwidth is below an available bandwidth threshold, bandwidth allocations of at least a portion of the plurality of compute resources to maximize an overall bandwidth usage of the shared memory pool as a weighted function of the set of compute resource priorities.
20. The method of claim 19 , further comprising:
generating a queue of the plurality of data requests for each compute resource priority; and
generating a data request sequence by arbitrating the data requests in the queues as a weighted function of the plurality of compute resource priorities.
21. The method of claim 20 , further comprising regulating bandwidth of data requests from a compute resource to modify a frequency of data requests from the compute resource in the data request sequence.
22. The method of claim 20 , further comprising regulating entry of data requests from a compute resource into the queue for the compute resource priority of the compute resource to modify a frequency of data requests from the compute resource in the data request sequence.