Systems and methods for dynamically choosing a processing element for a compute kernel
View Patent ↗A runtime system implemented in accordance with the present invention provides an application platform for parallel-processing computer systems. Such a runtime system enables users to leverage the computational power of parallel-processing computer systems to accelerate/optimize numeric and array-intensive computations in their application programs. This enables greatly increased performance of high-performance computing (HPC) applications.
1. A computer-implemented method, comprising:
at a runtime system executing at a parallel-processing computer that includes multiple types of processing elements using shared memory:
at runtime:
receiving one or more operation requests issued by an application;
choosing a respective type of processing element for at least one of the one or more operation requests, the at least one operation request corresponding to an intrinsic operation or a primitive operation, wherein the respective type of processing element is selected from the group consisting of single-core/multi-core central processing units, graphics processing units and single-core/multi-core co-processors;
identifying, from a library, a precompiled, processor-specific compute kernel corresponding to the intrinsic operation or the primitive operation of the at least one operation request, the library comprising a plurality of processor-specific compute kernels corresponding to a plurality of intrinsic operations or primitive operations; and
preparing one or more compute kernels for the at least one operation request, wherein the one or more compute kernels include the at least one precompiled, processor-specific compute kernel, and are configured to execute on the respective type of processing element.
2. The computer-implemented method of claim 1 , wherein the precompiled, processor-specific compute kernels in the library are compressed and/or encrypted.
3. The computer-implemented method of claim 1 , wherein the precompiled, processor-specific compute kernels are hand-coded for a specific type of processor.
4. The computer-implemented method of claim 1 , wherein the intrinsic operation corresponding to the at least one operation request is selected from a group consisting of: matrix multiplication, matrix solvers, fast Fourier transforms, convolutions and LU decomposition.
5. The computer-implemented method of claim 1 , wherein the primitive operation corresponding to the at least one operation request is selected from a group consisting of: arithmetic operations and trigonometric functions.
6. The computer-implemented method of claim 1 , wherein the respective type of processing element is chosen based, at least in part, on one of a predefined computer resource requirement metric of the one or more operation requests and a predefined workload requirement metric of the parallel-processing computer running the runtime system.
7. The computer-implemented method of claim 1 , wherein the one or more operation requests correspond to application program interface function calls.
8. The computer-implemented method of claim 1 , wherein the identified precompiled, processor-specific compute kernel corresponding to the intrinsic operation or the primitive operation is in a GPU binary instruction set architecture format.
9. A parallel-processing computer system, comprising:
memory;
multiple types of processing elements using shared memory; and
at least one program stored in the memory and executed by the multiple types of processing elements, the at least one program including a runtime system comprising instructions for:
at runtime:
receiving one or more operation requests issued by an application;
choosing a respective type of processing element for at least one of the one or more operation requests, the at least one operation request corresponding to an intrinsic operation or a primitive operation, wherein the respective type of processing element is selected from the group consisting of single-core/multi-core central processing units, graphics processing units and single-core/multi-core co-processors;
identifying, from a library, a precompiled, processor-specific compute kernel corresponding to the intrinsic operation or the primitive operation of the at least one operation request, the library comprising a plurality of processor-specific compute kernels corresponding to a plurality of intrinsic operations or primitive operations; and
preparing one or more compute kernels for the at least one operation request, wherein the one or more compute kernels include the at least one precompiled, processor-specific compute kernel, and are configured to execute on the respective type of processing element.
10. The parallel-processing computer system of claim 9 , wherein the precompiled, processor-specific compute kernels in the library are compressed and/or encrypted.
11. The parallel-processing computer system of claim 9 , wherein the precompiled, processor-specific compute kernels are hand-coded for a specific type of processor.
12. The parallel-processing computer system of claim 9 , wherein the intrinsic operation corresponding to the at least one operation request is selected from a group consisting of: matrix multiplication, matrix solvers, fast Fourier transforms, convolutions and LU decomposition.
13. The parallel-processing computer system of claim 9 , wherein the primitive operation corresponding to the at least one operation request is selected from a group consisting of: arithmetic operations and trigonometric functions.
14. The parallel-processing computer system of claim 9 , wherein the respective type of processing element is chosen based, at least in part, on one of a predefined computer resource requirement metric of the one or more operation requests and a predefined workload requirement metric of the parallel-processing computer running the runtime system.
15. The parallel-processing computer system of claim 9 , wherein the one or more operation requests correspond to application program interface function calls.
16. The parallel-processing computer system of claim 9 , wherein the identified precompiled, processor-specific compute kernel corresponding to the intrinsic operation or the primitive operation is in a GPU binary instruction set architecture format.
17. A non-transitory computer readable storage medium storing one or more programs comprising instructions for:
at runtime:
receiving one or more operation requests issued by an application;
choosing a respective type of processing element for at least one of the one or more operation requests, the at least one operation request corresponding to an intrinsic operation or a primitive operation, wherein the respective type of processing element is selected from the group consisting of single-core/multi-core central processing units, graphics processing units and single-core/multi-core co-processors;
identifying, from a library, a precompiled, processor-specific compute kernel corresponding to the intrinsic operation or the primitive operation of the at least one operation request, the library comprising a plurality of processor-specific compute kernels corresponding to a plurality of intrinsic operations or primitive operations; and
preparing one or more compute kernels for the at least one operation request, wherein the one or more compute kernels include the at least one precompiled, processor-specific compute kernel, and are configured to execute on the respective type of processing element; and
wherein the one or more programs are configured to be executed by a parallel-processing computer that includes multiple types of processing elements using shared memory.
18. The non-transitory computer readable storage medium of claim 17 , wherein the precompiled, processor-specific compute kernels in the library are compressed and/or encrypted.
19. The non-transitory computer readable storage medium of claim 17 , wherein the precompiled, processor-specific compute kernels are hand-coded for a specific type of processor.
20. The non-transitory computer readable storage medium of claim 17 , wherein the intrinsic operation corresponding to the at least one operation request is selected from a group consisting of: matrix multiplication, matrix solvers, fast Fourier transforms, convolutions and LU decomposition.
21. The non-transitory computer readable storage medium of claim 17 , wherein the primitive operation corresponding to the at least one operation request is selected from a group consisting of: arithmetic operations and trigonometric functions.
22. The non-transitory computer readable storage medium of claim 17 , wherein the respective type of processing element is chosen based, at least in part, on one of a predefined computer resource requirement metric of the one or more operation requests and a predefined workload requirement metric of the parallel-processing computer running the runtime system.
23. The non-transitory computer readable storage medium of claim 17 , wherein the one or more operation requests correspond to application program interface function calls.
24. The non-transitory computer readable storage medium of claim 17 , wherein the identified precompiled, processor-specific compute kernel corresponding to the intrinsic operation or the primitive operation is in a GPU binary instruction set architecture format.