Infrastructure managed workload distribution
System and techniques for infrastructure managed workload distribution are described herein. An infrastructure processing unit (IPU) receives a workload that includes a workload definition. The workload definition includes stages of the workload and a performance expectation. The IPU provides the workload, for execution, to a processing unit of a compute node to which the IPU belongs. The IPU monitors execution of the workload to determine that a stage of the workload is performing outside of the performance expectation from the workload definition. In response, the IPU modifies the execution of the workload.
1 . An infrastructure processing unit (IPU) comprising:
a network connector;
a Compute Express Link (CXL) connector; and
processing circuitry configured to:
receive a workload via the network connector or the CXL connector, the workload including a workload definition including a plurality of pipelined stages of the workload and a respective performance expectation for each of the plurality of pipelined stages;
provide, via the CXL connector, the workload to a processor of a compute node for execution;
access execution stacks from pooled memory of the compute node, wherein the execution stacks indicate that a stage of the workload is performing below its respective performance expectation based on the workload execution; and
modify the execution of the stage of the workload in response to the indication that the stage of the workload is performing below its respective performance expectation, wherein the modifying results in performance of the workload meeting a predetermined service level agreement by intercepting, during the execution and via the CXL connector, a resource request of the stage of the workload to the compute node and redirecting the resource request to additional hardware utilized by the stage of the workload during the execution.
2 . The IPU of claim 1 , wherein, to modify the execution of the stage of the workload, the processing circuitry is further configured to:
stop the stage of the workload on the compute node; and
transfer the stage of the workload to a second compute node via the network connector or the CXL connector.
3 . The IPU of claim 2 , wherein the second compute node is one of multiple compute nodes accessible via the network connector or via the CXL connector.
4 . The IPU of claim 3 , wherein the multiple compute nodes each have a corresponding resource availability.
5 . The IPU of claim 4 , wherein the processing circuitry is further configured to:
receive, at predefined intervals, resource messages from each of the multiple compute nodes, the resource messages indicating an available resource at a respective compute node; and
store records for each of the multiple compute nodes of the available resource from the resource messages.
6 . The IPU of claim 5 , wherein the records are stored according to a predetermined order.
7 . The IPU of claim 1 , wherein the IPU further comprises:
tracking circuitry;
migration circuitry;
pipeline circuitry; and
telemetry circuitry.
8 . The IPU of claim 1 , wherein the processing circuitry is further configured to receive, from the compute node, interrupts or instruction fetches associated with the execution of the workload indicating that the stage of the workload is performing below its respective performance expectation.
9 . The IPU of claim 1 , wherein the processing circuitry is further configured to store the workload definition in a memory of the IPU.
10 . The IPU of claim 9 , wherein the memory is a register, dynamic random access memory (DRAM), synchronous DRAM (SDRAM), or static RAM (SRAM).
11 . The IPU of claim 1 , wherein the respective performance expectation is a respective expected time to complete the execution of the respective stage, and wherein performing below the respective performance expectation includes taking longer to complete the respective stage than the respective expected time to complete included in the workload definition.
12 . The IPU of claim 1 , wherein the workload definition includes a workload identification (ID) field, a stage ID field, an expected performance field, and a resource ID field.
13 . At least one non-transitory machine readable medium including instructions that when executed by processing circuitry, cause the processing circuitry to execute operations comprising:
receiving, at an infrastructure processing unit (IPU) and via a network connector or a CXL connector of a compute node, a workload including a workload definition including a plurality of pipelined stages of the workload and a respective performance expectation for each of the plurality of pipelined stages;
providing, via the CXL connector, the workload to a processor of the compute node for execution;
accessing execution stacks from pooled memory of the compute node, wherein the execution stacks indicate that a stage of the workload is performing below its respective performance expectation based on the workload execution; and
modifying the execution of the stage of the workload in response to the indication that the stage of the workload is performing below its respective performance expectation, wherein the modifying results in performance of the workload meeting a predetermined service level agreement by intercepting, during the execution and via the CXL connector, a resource request of the stage of the workload to the compute node and redirecting the resource request to additional hardware utilized by the stage of the workload during the execution.
14 . The at least one non-transitory machine readable medium of claim 13 , wherein modifying the execution of the stage of the workload further includes:
stopping the stage of the workload on the compute node; and
transferring the stage of the workload to a second compute node via the network connector or the CXL connector.
15 . The at least one machine readable medium of claim 14 , wherein the second compute node is one of multiple compute nodes accessible via the network connector or via the CXL connector.
16 . The at least one non-transitory machine readable medium of claim 15 , wherein the multiple compute nodes each have a corresponding resource availability.
17 . The at least one non-transitory machine readable medium of claim 16 , wherein the operations further comprise:
receiving, periodically, resource messages from each of the multiple compute nodes, the resource messages indicating an available resource at a respective compute node; and
storing records for each of the multiple compute nodes of the available resource from the resource messages.
18 . The at least one non-transitory machine readable medium of claim 17 , wherein the records are stored according to a predetermined order.
19 . The at least one non-transitory machine readable medium of claim 13 , wherein the IPU further comprises:
tracking circuitry;
migration circuitry;
pipeline circuitry; and
telemetry circuitry.
20 . The at least one non-transitory machine readable medium of claim 13 , wherein the operations further comprise receiving, from the compute node, interrupts or instruction fetches associated with the execution of the workload indicating that the stage of the workload is performing below its respective performance expectation.
21 . The at least one non-transitory machine readable medium of claim 13 , wherein the operations further comprise storing the workload definition in a memory of the IPU.
22 . The at least one non-transitory machine readable medium of claim 21 , wherein the memory is a register, dynamic random access memory (DRAM), synchronous DRAM (SDRAM), or static RAM (SRAM).
23 . The at least one non-transitory machine readable medium of claim 13 , wherein the respective performance expectation is a respective expected time to complete the execution of the respective stage, and wherein performing below the respective performance expectation includes taking longer to complete the respective stage than the respective expected time to complete included in the workload definition.
24 . The at least one non-transitory machine readable medium of claim 13 , wherein the workload definition includes a workload identification (ID) field, a stage ID field, an expected performance field, and a resource ID field.