Inter-baseboard management controller (BMC) integration for high performance computing platforms
Embodiments of systems and methods for inter-Baseboard Management Controller (BMC) integration for High Performance Computing (HPC) platforms are described. In some embodiments, an HPC platform may include a first BMC coupled to a host processor; and a hardware accelerator comprising: (a) one or more managed subsystems coupled to the host processor, and (b) a second BMC coupled to the one or more managed subsystems and decoupled from the host processor, wherein the first BMC and the second BMC are coupled to each other via a high-speed, Out-of-Band (OOB) management link.
1. A High Performance Computing (HPC) platform, comprising:
a first Baseboard Management Controller (BMC) coupled to a host processor; and
a hardware accelerator comprising: (a) one or more managed subsystems coupled to the host processor, and (b) a second BMC coupled to the one or more managed subsystems and decoupled from the host processor, wherein the first BMC and the second BMC are coupled to each other via an Out-of-Band (OOB) management link.
2. The HPC platform of claim 1 , wherein the one or more managed subsystems comprise at least one of: a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), an Intelligence Processing Unit (IPU), a Data Processing Unit (DPU), a Gaussian Neural Accelerator (GNA), an Audio and Contextual Engine (ACE), or a Vision Processing Unit (VPU).
3. The HPC platform of claim 1 , wherein the one or more managed subsystems are mounted on an accelerator tray.
4. The HPC platform of claim 1 , wherein the high-speed, OOB management link comprises a Peripheral Component Interconnect Express (PCIe) link.
5. The HPC platform of claim 4 , wherein the first BMC is configured to communicate with the second BMC over the PCIe link using Management Component Transport Protocol (MCTP) over PCIe Vendor-Defined Messages (VDM).
6. The HPC platform of claim 4 , wherein the first BMC comprises a PCIe Root Complex and the second BMC comprises a PCIe Endpoint coupled to the PCIe Root Complex.
7. The HPC platform of claim 6 , wherein the second BMC further comprises a BMC MCTP Endpoint coupled to the PCIe Endpoint via an MCTP bridge.
8. The HPC platform of claim 7 , wherein the one or more managed subsystems are coupled to the PCIe Endpoint and to the BMC MCTP Endpoint via the MCTP bridge.
9. The HPC platform of claim 8 , wherein the first BMC comprises program instructions stored thereon that, upon execution, cause the first BMC to access the one or more managed subsystems via a PCIe connection while bypassing the second BMC.
10. The HPC platform of claim 6 , wherein the one or more managed subsystems are coupled to a BMC MCTP Endpoint via a local management MCTP Bus Owner.
11. The HPC platform of claim 10 , wherein the first BMC comprises program instructions stored thereon that, upon execution, cause the first BMC to access the one or more managed subsystems via the PCIe link.
12. The HPC platform of claim 1 , wherein the first BMC is configured to assign the second BMC one or more management tasks.
13. The HPC platform of claim 12 , wherein the one or more management tasks comprise at least one of: collection or processing telemetry data from the one or more managed subsystems.
14. The HPC platform of claim 12 , wherein the one or more management tasks comprise deployment of a firmware update with respect to one or more managed subsystems.
15. A Baseboard Management Controller (BMC) coupled to a host processor of a High Performance Computing (HPC) platform, the BMC having program instructions stored thereon that, upon execution, cause the BMC to:
establish a Management Component Transport Protocol (MCTP) over Peripheral Component Interconnect Express (PCIe) Vendor-Defined Messages (VDM) link with another BMC coupled to one or more accelerator components; and
communicate with the other BMC, at least in part, via the MCTP over PCIe VDM link.
16. The BMC of claim 15 , wherein the one or more accelerator components comprise one or more of: a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), an Intelligence Processing Unit (IPU), a Data Processing Unit (DPU), a Gaussian Neural Accelerator (GNA), an Audio and Contextual Engine (ACE), or a Vision Processing Unit (VPU).
17. The BMC of claim 15 , wherein the program instructions, upon execution, cause the BMC to communicate with the other BMC via an Operating System (OS) agent in response to a determination that the MCTP over PCIe VDM link is not operational.
18. A method, comprising:
establishing, by a tray Baseboard Management Controller (BMC) of an accelerator tray having one or more accelerator cores coupled thereto, a Management Component Transport Protocol (MCTP) over Peripheral Component Interconnect Express (PCIe) Vendor-Defined Messages (VDM) link with a system BMC coupled to a host processor of a High Performance Computing (HPC) platform; and
communicating with the system BMC, at least in part, via the MCTP over PCIe VDM link.
19. The method of claim 18 , further comprising providing, by the tray BMC to the system BMC, direct access to a first subset of the accelerator cores and indirect access to a second subset of the accelerator cores.
20. The method of claim 18 , further comprising, in response to a determination the MCTP over PCIe VDM link is not operational, communicating with the system BMC via an Operating System (OS) agent.