Compiler optimization for many integrated core processors
Systems and methods for source-to-source transformation for compiler optimization for many integrated core (MIC) coprocessors, including identifying data dependencies in candidate loops and data elements used in each iteration for arrays, profiling candidate loops to find a proper number m, wherein data transfer and computation for m iterations take an equal amount of time, and creating an outer loop outside the candidate loop, with each iteration of the outer loop executing m iterations of the candidate loop. Data streaming is performed by determining optimum buffer size for one or more arrays and inserting code before the outer loop to create optimum sized buffers, overlapping data transfer between central processing units (CPUs) and MICs with the computation; reusing buffers to reduce memory employed on the MICs, and reusing threads on MICs to repeatedly launch kernels on the MICs for asynchronous data transfer.
1. A method for source-to-source transformation for compiler optimization for one or more many integrated core (MIC) coprocessors, comprising:
identifying data dependencies in one or more candidate loops and data elements used in each iteration for one or more arrays;
profiling the one or more candidate loops to find a proper number m, wherein data transfer and computation for m iterations take an equal amount of time;
creating an outer loop outside the candidate loop, wherein each iteration of the outer loop executes m iterations of the candidate loop; and
performing data streaming, wherein the data streaming comprises:
determining optimum buffer size for one or more arrays, and inserting code before the outer loop to create one or more optimum sized buffers;
overlapping data transfer between one or more central processing units (CPUs) and the MICs with the computation to hide data transfer overload;
reusing the buffers to reduce memory employed on the MICs during the data transfer; and
reusing threads on the MICs to repeatedly launch kernels on the MICs for asynchronous data transfer.
2. The method as recited in claim 1 , further comprising executing code segments on the MICs when an entire data set is too large to be held in memory of the MICs.
3. The method as recited in claim 1 , wherein data is transferred in a plurality of small blocks, the small blocks being smaller than an entire chunk of the data.
4. The method as recited in claim 1 , wherein code is inserted before the outer loop to asynchronously copy a first segment of an array to a first MIC buffer.
5. The method as recited in claim 1 , further comprising inserting code to free all buffers after the outer loop.
6. The method as recited in claim 1 , further comprising:
enabling direct memory access for transferring pointer-based structures between the CPU and the MIC; and
optimizing memory utilization of the MIC by allocating free memory space on the MIC for other use when a data structure size is smaller than memory space on the MIC, and employing all memory space on the MIC when a data structure size approaches the memory space on the MIC.
7. A method for source-to-source transformation for compiler optimization for one or more many integrated core (MIC) coprocessors, comprising:
regularizing irregular memory on a loop head of a candidate loop by reordering computations, and adjusting array access according to the regularized irregular memory to enable data streaming and vectorization on the one or more MICs during data transfer;
identifying all irregular memory array accesses in a candidate loop that do not use a loop index i as an array index;
regularizing irregular memory array accesses automatically, wherein array indices employed in all memory access is set to the loop index i;
hiding regularization overhead by overlapping regularization with the data transfer and the computations when data streaming is employed;
determining optimum buffer size for one or more arrays, and inserting code before an outer loop outside the candidate loop to create one or more optimum sized buffers;
overlapping data transfer between one or more central processing units (CPUs) and the MICs with a computation to hide data transfer overload;
reusing the buffers to reduce memory employed on the MICs during the data transfer; and
reusing threads on the MICs to repeatedly launch kernels on the MICs for asynchronous data transfer.
8. The method as recited in claim 7 , wherein the candidate loop is split into a first and a second loop, wherein irregular accesses are split into the first loop, and regular accesses are split into the second loop.
9. The method as recited in claim 8 , wherein the data streaming and vectorization is performed on the second loop after the candidate loop is split.
10. The method as recited in claim 7 , further comprising creating threads on a central processing unit (CPU) to perform operations in parallel with the candidate loop.
11. The method as recited in claim 7 , further comprising:
enabling direct memory access for transferring pointer-based structures between a CPU and the MIC; and
optimizing memory utilization of the MIC by allocating free memory space on the MIC for other use when a data structure size is smaller than memory space on the MIC, and employing all memory space on the MIC when a data structure size approaches the memory space on the MIC.
12. A system for source-to-source transformation for compiler optimization for one or more many integrated core (MIC) coprocessors, comprising:
an identification module configured to identify data dependencies in one or more candidate loops and data elements used in each iteration for one or more arrays;
a profiling module configured to profile the one or more candidate loops to find a proper number m, wherein data transfer and computation for m iterations take an equal amount of time;
a loop creation module configured to create an outer loop outside the candidate loop, wherein each iteration of the outer loop executes m iterations of the candidate loop;
a data streaming module configured to perform data streaming, wherein the data streaming comprises:
determining optimum buffer size for one or more arrays, and inserting code before the outer loop to create one or more optimum sized buffers;
overlapping data transfer between one or more central processing units (CPUs) and the MICs with the computation to hide data transfer overload;
reusing the buffers to reduce memory employed on the MICs during the data transfer; and
reusing threads on the MICs to repeatedly launch kernels on the MICs for asynchronous data transfer.
13. The system as recited in claim 12 , further comprising executing code segments on the MICs when an entire data set is too large to be held in memory of the MICs.
14. The system as recited in claim 12 , wherein data is transferred in a plurality of small blocks, the small blocks being smaller than an entire chunk of the data.
15. The system as recited in claim 12 , wherein code is inserted before the outer loop to asynchronously copy a first segment of an array to a first MIC buffer.
16. The system as recited in claim 12 , wherein code is inserted to free all buffers after the outer loop.
17. The system as recited in claim 12 , wherein direct memory access is employed to transfer pointer-based structures between the CPU and the MIC, and memory utilization of the MIC is optimized by allocating free memory space on the MIC for other use when a data structure size is smaller than memory space on the MIC, and employing all memory space on the MIC when a data structure size approaches the memory space on the MIC.