Efficient accelerator offload in multi-accelerator framework
Methods, apparatus, and software for efficient accelerator offload in multi-accelerator frameworks. One multi-accelerator framework employs a compute platform including a plurality of processor cores and a plurality of accelerator devices. An application is executed on a first core and a portion of the application workload is offloaded to a first accelerator device. In connection with moving execution of the application to a second core, a second accelerator devices to be used for the offloaded workload is selected based on core-to-accelerator cost information for the second core. This core-to-accelerator cost information includes core-accelerator cost information for combinations of core-accelerator pairs, which are based, at least on part, on latencies projected for interconnect paths between cores and accelerators. Both single-socket and multi-socket platform are supported. The solutions include mechanisms for moving offloaded workloads for multiple accelerator devices, as well as synchronizing accelerator operations and workflows.
1. A method implemented in a compute platform including a plurality of processor cores and a plurality of accelerator devices, comprising:
executing an application on a first core;
offloading a first offloaded portion of a workload for the application to a first accelerator device;
moving execution of the application from the first core to a second core;
selecting a second accelerator device to be used based on core-to-accelerator cost information for the second core; and
offloading the first offloaded portion of the workload to the second accelerator device,
wherein the core-to-accelerator cost information for the second core is based, at least on part, on latencies projected for interconnect paths between core-accelerator pairs including the second core and accelerators among the plurality of accelerators.
2. The method of claim 1 , further comprising:
storing core-to-accelerator cost information in at least one data structure on the compute platform;
in connection with moving execution of the application from the first core to the second core, sending a request message including an identity of the second core to an entity enabled to access the at least one data structure; and
one of,
returning a response message including cost-to-accelerator cost information for that second core; or
returning a response message identifying an accelerator having a lowest core-to-accelerator metric for the second core.
3. The method of claim 2 , further comprising:
calculating core-to-accelerator costs for multiple core-accelerator device pairs based on current platform core workloads, and
updating the core-to-accelerator costs in the at least one data structure for the multiple core-accelerator device pairs.
4. The method of claim 2 , wherein the entity enabled to access the at least one data structure comprises platform BIOS.
5. The method of claim 2 , wherein the compute platform includes support for Advanced Configuration and Power Interface (ACPI) functions, and wherein the request message is an ACPI system call.
6. The method of claim 1 , wherein the compute platform is a multi-socket platform including multiple sockets, each having a plurality of processor cores, and wherein the first core is on a first socket and the second core is on a second socket.
7. The method of claim 6 , wherein the multi-socket platform is a Non-Uniform Memory Architecture (NUMA) platform, wherein the first offloaded portion of a workload has associated data that are stored in memory on the first socket, and wherein the data associated with the first offloaded portion of the workload remain in the memory on the first socket when the first offloaded portion of the workload is offloaded to the second accelerator device.
8. The method of claim 1 , further comprising:
employing transmit and receive rings for the first and second accelerator devices; and
synchronizing the transmit and receive rings for the first and second accelerator devices.
9. The method of claim 1 , further comprising:
while executing the application on the first core, offloading a second offloaded portion of the workload for the application to a third accelerator device;
in connection with moving execution of the application from the first core to the second core, selecting a fourth accelerator device to be used based on core-to-accelerator cost information for the second core; and
offloading the second offloaded portion of the workload to the fourth accelerator device.
10. The method of claim 9 , wherein the first and third accelerator devices are implemented in a workflow and the second and fourth accelerator devices are implemented in the workflow further comprising synchronizing the states of the first and second accelerator devices and the states of the third and fourth accelerator devices.
11. A non-transitory machine-readable medium having instructions stored thereon, which are configured to be executed on at least one processor core in a compute platform having a plurality of processor cores and a plurality of accelerator devices, to enable the compute platform to:
assigning execution of an application to a first core;
assigning a first accelerator device to be used to process a first offloaded portion of a workload for the application that is offloaded to the first accelerator device;
moving execution of the application from the first core to a second core;
selecting a second accelerator device to be used based on core-to-accelerator cost information for the second core; and
assigning the second accelerator device to be used for processing the first offloaded portion of the workload for the application,
wherein the core-to-accelerator cost information for the second core is based, at least on part, on latencies projected for interconnect paths between core-accelerator pairs including the second core and accelerators among the plurality of accelerators.
12. The non-transitory machine-readable medium of claim 11 , wherein execution of the instructions further enables the compute platform to:
in connection with moving execution of the application from the first core to the second core, send a request message including an identity of the second core to an entity enabled to access at least one data structure in which core-to-accelerator cost information is stored; and
one of,
receive a response message including cost-to-accelerator cost information for that second core; or
receive a response message identifying an accelerator having a lowest core-to-accelerator metric for the second core.
13. The non-transitory machine-readable medium of claim 12 , wherein the compute platform includes support for Advanced Configuration and Power Interface (ACPI) functions, and wherein the request message is an ACPI system call (syscall).
14. The non-transitory machine-readable medium of claim 11 , wherein the compute platform includes at least one data structure in which core-to-accelerator cost information is stored, and wherein execution of the instructions further enables the compute platform to:
calculate core-to-accelerator costs for multiple core-accelerator device pairs based on current platform core workloads, and
update core-to-accelerator costs in the at least one data structure for the multiple core-accelerator device pairs.
15. The non-transitory machine-readable medium of claim 11 , wherein the compute platform is a multi-socket platform including multiple sockets, each having a plurality of processor cores, and wherein the first core is on a first socket and the second core is on a second socket.
16. A compute platform, comprising:
a plurality of processor cores, operatively coupled to at least one memory; and
a plurality of accelerator devices,
wherein the compute platform is configured to,
execute an application on a first core;
offload a first offloaded portion of a workload for the application to a first accelerator device;
move execution of the application from the first core to a second core;
select a second accelerator device to be used based on core-to-accelerator cost information for the second core; and
offload the first offloaded portion of the workload to the second accelerator device,
wherein the core-to-accelerator cost information for the second core is based, at least on part, on latencies projected for interconnect paths between core-accelerator pairs including the second core and accelerators among the plurality of accelerators.
17. The compute platform of claim 16 , wherein the compute platform is further configured to:
store core-to-accelerator cost information in at least one data structure;
in connection with moving execution of the application from the first core to the second core, send a request message including an identity of the second core to an entity enabled to access the at least one data structure; and
one of,
return a response message including cost-to-accelerator cost information for that second core; or
return a response message identifying an accelerator having a lowest core-to-accelerator metric for the second core.
18. The compute platform of claim 17 , wherein the compute platform includes support for Advanced Configuration and Power Interface (ACPI) functions, and wherein the request message is an ACPI system call.
19. The compute platform of claim 16 , wherein the compute platform is a multi-socket platform including multiple sockets, each having a plurality of processor cores, and wherein the first core is on a first socket and the second core is on a second socket.
20. The compute platform of claim 16 , wherein the compute platform is further configured to:
while executing the application on the first core, offload a second offloaded portion of the workload for the application to a third accelerator device;
in connection with moving execution of the application from the first core to the second core, selecting a fourth accelerator device to be used based on core-to-accelerator cost information for the second core; and
offload the second offloaded portion of the workload to the fourth accelerator device.