Methods and apparatus for runtime multi-scheduling of software executing on a heterogeneous system

1. An apparatus for runtime scheduling of software executing on a heterogeneous system, the apparatus comprising:

a compilation auto-scheduler to generate a schedule based on configurations for processing elements of the heterogeneous system, the compilation autoscheduler to generate the schedule in response to a variant compiler generating a representation of an algorithm in a domain-specific language (DSL), the processing elements including at least a first processing element and a second processing element;

the variant compiler to compile variant binaries based on the schedule, each of the variant binaries associated with the algorithm in the DSL, the variant binaries including a first variant binary corresponding to the first processing element and a second variant binary corresponding to the second processing element;

an application compiler to generate a fat binary including a runtime scheduler to select one or more of the variant binaries to execute a workload based on the schedule; and

at least one logic circuit to implement at least one of the compilation auto-scheduler, the variant compiler, or the application compiler.

2. The apparatus of claim 1 , wherein the configurations include a first configuration corresponding to the first processing element and a second configuration corresponding to the second processing element, the first configuration including at least one of a host architecture of the first processing element, an application programming interface (API) associated with the first processing element, or scheduling heuristics known to improve a performance of the first processing element.

3. The apparatus of claim 1 , wherein the processing elements include at least one of a central processing unit, a field programmable gate array, a graphics processing unit, or a vision processing unit.

4. The apparatus of claim 1 , wherein:

the variant compiler is to:

add the first variant binary to a variant library;

add a jump table entry to a jump table, the jump table entry including a variant symbol associated with the first processing element;

generate a jump table library including the jump table; and

generate an application programming interface (API) to invoke the runtime scheduler to access the jump table; and

the application compiler is to generate an executable by linking the executable and the fat binary, the executable to invoke the runtime scheduler by invoking the API.

5. The apparatus of claim 1 , further including:

a feedback interface to obtain a performance characteristic of the heterogeneous system from the fat binary, the performance characteristic associated with one or more of the processing elements executing the workload at a first runtime, the fat binary executing according to a function designating successful execution of the fat binary on the heterogeneous system; and

a performance analyzer to:

determine a performance delta based on the performance characteristic and the function; and

prior to a second runtime, adjusting, using a machine learning model, a cost model of the first processing element based on the performance delta.

6. The apparatus of claim 5 , wherein the cost model is a first cost model, and further including a cost model learner to, prior to the second runtime, by using a neural network, adjust a second cost model of the second processing element based on the performance delta.

7. The apparatus of claim 5 , wherein the performance analyzer is to determine the performance delta by determining a difference between performance achieved at the first runtime and performance as defined by the function designating successful execution of the fat binary on the heterogeneous system.

8. A non-transitory computer readable storage medium comprising instructions that, when executed, cause a machine to:

generate a representation of an algorithm in a domain-specific language (DSL);

generate a schedule based on configurations for processing elements of a heterogeneous system, the processing elements including at least a first processing element and a second processing element;

compile variant binaries based on the schedule, each of the variant binaries associated with the algorithm in the DSL, the variant binaries including a first variant binary corresponding to the first processing element and a second variant binary corresponding to the second processing element; and

generate a fat binary including a runtime scheduler to select one or more of the variant binaries to execute a workload based on the schedule.

9. The non-transitory computer readable storage medium of claim 8 , wherein the configurations include a first configuration corresponding to the first processing element and a second configuration corresponding to the second processing element, the first configuration including at least one of a host architecture of the first processing element, an application programming interface (API) associated with the first processing element, or scheduling heuristics known to improve a performance of the first processing element.

10. The non-transitory computer readable storage medium of claim 8 , wherein the processing elements include at least one of a central processing unit, a field programmable gate array, a graphics processing unit, or a vision processing unit.

11. The non-transitory computer readable storage medium of claim 8 , wherein the instructions, when executed, cause the machine to:

add the first variant binary to a variant library;

add a jump table entry to a jump table, the jump table entry including a variant symbol associated with the first processing element;

generate a jump table library including the jump table;

generate an application programming interface (API) to invoke the runtime scheduler to access the jump table; and

generate an executable by linking the executable and the fat binary, the executable to invoke the runtime scheduler by invoking the API.

12. The non-transitory computer readable storage medium of claim 8 , wherein the instructions, when executed, cause the machine to:

obtain a performance characteristic of the heterogeneous system from the fat binary, the performance characteristic associated with one or more of the processing elements executing the workload at a first runtime, the fat binary executing according to a function designating successful execution of the fat binary on the heterogeneous system;

determine a performance delta based on the performance characteristic and the function; and

prior to a second runtime, adjust, using a machine learning model, a cost model of the first processing element based on the performance delta.

13. The non-transitory computer readable storage medium of claim 12 , wherein the cost model is a first cost model, and wherein the instructions, when executed, cause the machine to, prior to the second runtime, using a neural network, adjust a second cost model of the second processing element based on the performance delta.

14. The non-transitory computer readable storage medium of claim 12 , wherein the instructions, when executed, cause the machine to determine the performance delta by determining a difference between performance achieved at the first runtime and performance as defined by the function designating successful execution of the fat binary on the heterogeneous system.

15. A method for runtime scheduling of software executing on a heterogeneous system, the method comprising:

generating a representation of an algorithm in a domain-specific language (DSL);

generating a schedule based on configurations for processing elements of the heterogeneous system, the processing elements including at least a first processing element and a second processing element;

compiling variant binaries based on the schedule, each of the variant binaries associated with the algorithm in the DSL, the variant binaries including a first variant binary corresponding to the first processing element and a second variant binary corresponding to the second processing element; and

generating a fat binary including a runtime scheduler to select one or more of the variant binaries to execute a workload based on the schedule.

16. The method of claim 15 , wherein the configurations include a first configuration corresponding to the first processing element and a second configuration corresponding to the second processing element, the first configuration including at least one of a host architecture of the first processing element, an application programming interface (API) associated with the first processing element, or scheduling heuristics known to improve a performance of the first processing element.

17. The method of claim 15 , wherein the processing elements include at least one of a central processing unit, a field programmable gate array, a graphics processing unit, or a vision processing unit.

18. The method of claim 15 , further including:

adding the first variant binary to a variant library;

adding a jump table entry to a jump table, the jump table entry including a variant symbol associated with the first processing element;

generating a jump table library including the jump table;

generating an application programming interface (API) to invoke the runtime scheduler to access the jump table; and

generating an executable by linking the executable and the fat binary, the executable to invoke the runtime scheduler by invoking the API.

19. The method of claim 15 , further including:

obtaining a performance characteristic of the heterogeneous system from the fat binary, the performance characteristic associated with one or more of the processing elements executing the workload at a first runtime, the fat binary executing according to a function designating successful execution of the fat binary on the heterogeneous system;

determining a performance delta based on the performance characteristic and the function; and

prior to a second runtime, adjusting, using a machine learning model, a cost model of the first processing element based on the performance delta.

20. The method of claim 19 , wherein the cost model is a first cost model, and further including prior to the second runtime, using a neural network, adjusting a second cost model of the second processing element based on the performance delta.

21. The method of claim 19 , wherein determining the performance delta includes determining a difference between performance achieved at the first runtime and performance as defined by the function designating successful execution of the fat binary on the heterogeneous system.

22. An apparatus for runtime scheduling of software executing on a heterogeneous system, the apparatus comprising:

first means for generating a representation of an algorithm in a domain-specific language (DSL);

second means for generating a schedule based on configurations for processing elements of the heterogeneous system, the processing elements including at least a first processing element and a second processing element;

means for compiling variant binaries based on the schedule, each of the variant binaries associated with the algorithm in the DSL, the variant binaries including a first variant binary corresponding to the first processing element and a second variant binary corresponding to the second processing element; and

third means for generating a fat binary including a runtime scheduler to select one or more of the variant binaries to execute a workload based on the schedule.

23. The apparatus of claim 22 , wherein the configurations include a first configuration corresponding to the first processing element and a second configuration corresponding to the second processing element, the first configuration including at least one of a host architecture of the first processing element, an application programming interface (API) associated with the first processing element, or scheduling heuristics known to improve a performance of the first processing element.

24. The apparatus of claim 22 , wherein the processing elements include at least one of a central processing unit, a field programmable gate array, a graphics processing unit, or a vision processing unit.

25. The apparatus of claim 22 , wherein:

the means for compiling is to:

add the first variant binary to a variant library;

add a jump table entry to a jump table, the jump table entry including a variant symbol associated with the first processing element;

generate a jump table library including the jump table; and

generate an application programming interface (API) to invoke the runtime scheduler to access the jump table; and

the third means for generating is to generate an executable by linking the executable and the fat binary, the executable to invoke the runtime scheduler by invoking the API.

26. The apparatus of claim 22 , further including:

means for obtaining a performance characteristic of the heterogeneous system from the fat binary, the performance characteristic associated with one or more of the processing elements executing the workload at a first runtime, the fat binary executing according to a function designating successful execution of the fat binary on the heterogeneous system;

means for determining a performance delta based on the performance characteristic and the function; and

means for adjusting, prior to a second runtime, using a machine learning model, a cost model of the first processing element based on the performance delta.

27. The apparatus of claim 26 , wherein the cost model is a first cost model, the means for adjusting is first means for adjusting, and further including second means for adjusting, prior to the second runtime, by using a neural network, a second cost model of the second processing element based on the performance delta.

28. The apparatus of claim 26 , wherein the means for determining is to determine the performance delta by determining a difference between performance achieved at the first runtime and performance as defined by the function designating successful execution of the fat binary on the heterogeneous system.