Technologies for hardware microservices accelerated in XPU
Methods, apparatus, and software and for hardware microservices accelerated in other processing units (XPUs). The apparatus may be a platform including a System on Chip (SOC) and an XPU, such as a Field Programmable Gate Array (FPGA). The FPGA is configured to implement one or more Hardware (HW) accelerator functions associated with HW microservices. Execution of microservices is split between a software front-end that executes on the SOC and a hardware backend comprising the HW accelerator functions. The software front-end offloads a portion of a microservice and/or associated workload to the HW microservice backend implemented by the accelerator functions. An XPU or FPGA proxy is used to provide the microservice front-ends with shared access to HW accelerator functions, and schedules/multiplexes access to the HW accelerator functions using, e.g., telemetry data generated by the microservice front-ends and/or the HW accelerator functions. The platform may be an infrastructure processing unit (IPU) configured to accelerate infrastructure operations.
1 . A platform comprising:
a circuit board to which multiple components are operatively coupled, including,
a System On Chip (SOC), having a plurality of cores;
an Other Processing Unit (XPU), communicatively coupled to the SOC via interconnect circuitry in the circuit board;
an input/output (I/O) interface coupled to at least one of the SOC and XPU; and
memory, in which host software to be executed on one or more of the plurality of cores in the SOC is stored, including software code to implement one or more hardware (HW) microservice front-ends,
wherein the XPU is configured to implement one or more accelerator functions used to accelerate HW microservice backend operations that are offloaded from the one or more HW microservice front-ends, and
wherein the platform is configured to be coupled, via the I/O interface, to a compute host including a central processing unit (CPU) used to host software-based workloads.
2 . The platform of claim 1 , wherein the host software includes an XPU proxy that is configured to enable HW microservices front-ends to share the one or more accelerator functions.
3 . The platform of claim 2 , wherein the XPU proxy is configured to:
receive telemetry data generated from at least one of the HW microservice front-ends and the one or more accelerator functions; and
schedule access to the accelerator functions by the HW microservice front-ends based, at least in part, on the telemetry data.
4 . The platform of claim 2 , wherein the host software includes software code for implementing a software microservice, and wherein the XPU proxy is configured to determine that a microservice should be implemented as a software microservice or a HW microservice.
5 . The platform of claim 1 , wherein the XPU comprises a Field Programmable Gate Array (FPGA), and the one or more accelerated functions comprise FPGA kernels.
6 . The platform of claim 5 , wherein the host software is configured to:
provision accelerator functions in the FPGA by sending FPGA kernel bitstreams to the FPGA; and
reprovision FPGA bits for at least one accelerator function by sending a new FPGA kernel bitstream to the FPGA.
7 . The platform of claim 1 , wherein the platform comprises an infrastructure processing unit (IPU).
8 . The platform of claim 7 , wherein the XPU comprises a Field Programmable Gate Array (FPGA), wherein the IPU comprises a Peripheral Component Interconnect Express (PCIe) card, and the FPGA is configured to implement one or more PCIe interfaces, one of which is the I/O interface.
9 . The platform of claim 1 , wherein the platform is configured to be installed in a server comprising the compute host.
10 . A method implemented on a platform including a System on Chip (SOC) operatively coupled to a circuit board and communicatively coupled to an other processing unit (XPU) operatively coupled to the circuit board, comprising:
configuring the XPU to implement one or more accelerator functions;
splitting execution of a microservice between a hardware front-end and a hardware backend, wherein the hardware front-end is executed via execution of software on the SOC, and the hardware backend is used to execute an offloaded portion of the microservice on an accelerator function in the XPU,
wherein the platform is coupled to a compute host having a central processing unit (CPU) hosting software-based workloads.
11 . The method of claim 10 , wherein the XPU comprises a Field Programmable Gate Array (FPGA), further comprising programming an accelerator kernel in the FPGA to implement an accelerator function.
12 . The method of claim 10 , further comprising:
executing a plurality of hardware front-ends on the SOC; and
sharing the one or more accelerator functions among the plurality of hardware front-ends.
13 . The method of claim 12 , further comprising:
obtaining telemetry data generated from at least one of the hardware front-ends and the one or more accelerator functions; and
scheduling access to the accelerator functions by the hardware front-ends based, at least in part, on the telemetry data.
14 . The method of claim 10 , wherein the platform comprises an infrastructure processing unit (IPU) and the XPU comprises a Field Programmable Gate Array (FPGA).
15 . The method of claim 10 , wherein the method is implemented on a platform in a data center, and wherein the hardware front-end is run in a server worker pod in the data center.
16 . A non-transitory machine-readable medium having software instructions stored thereon configured to be executed on one or more cores on a System on Chip (SOC) in a platform including the SOC and an other processing unit (XPU) communicatively coupled to the SOC, the XPU configured to implement one or more accelerator functions, wherein execution of the instructions on the SOC enables the platform to:
implement a hardware (HW) microservice front-end that performs a first portion of a microservice;
determine a second portion of the microservice to offload to a first accelerator function; and
offload the second portion of the microservice to the XPU to execute the first accelerator function.
17 . The non-transitory machine-readable medium of claim 16 , wherein execution of the software instructions enables the platform to implement a plurality of HW microservice front-ends, wherein the software instructions include a software instruction for implementing an XPU proxy service, wherein the XPU proxy service facilitates sharing of the one or more accelerator functions among the plurality of HW microservice front-ends.
18 . The non-transitory machine-readable medium of claim 17 , wherein execution of the software instructions further enabled the platform to:
receive telemetry data generated from at least one of the HW microservice front-ends and the one or more accelerator functions; and
schedule access to an accelerator function by a HW microservice front-end based, at least in part, on the telemetry data.
19 . The non-transitory machine-readable medium of claim 17 , wherein execution of the software instructions enables the XPU proxy service to:
schedule access to the one or more accelerator functions based on pre-existing characteristics of the one or more accelerator functions and feedback from one or more of the plurality of HW microservice front-ends.
20 . The non-transitory machine-readable medium of claim 16 , wherein the XPU comprises a Field Programmable Gate Array (FPGA), and wherein execution of the software instructions further enables the platform to:
provision accelerator functions in the FPGA by sending FPGA kernel bitstreams to the FPGA; and
reprovision FPGA bits for at least one accelerator function by sending a new FPGA kernel bitstream to the FPGA.