Preprocessing tensor operations for optimal compilation
Techniques to preprocess tensor operations prior to code generation to optimize compilation are disclosed. A computer readable representation of a linear algebra or tensor operation is received. A code transformation software component performs transformations include output reduction and fraction removal. The result is a set of linear equations of a single variable with integer coefficients. Such a set lends itself to more efficient code generation during compilation by a code generation software component. Use cases disclosed include targeting a machine learning hardware accelerator, receiving code in the form of an intermediate language generated by a cross-compiler with multiple front ends supporting multiple programming languages, and cloud deployment and execution scenarios.
1. A method to generate computer readable instructions, the method comprising:
transforming, by executing an instruction with at least one processor of a computing device, using an output reduction transformation, a computer language representation of a mathematical operation to create a first transformed representation of the mathematical operation, the first transformed representation including a set of one or more polynomials that are linear in a single variable and semantically equivalent to the computer language representation;
transforming, by executing an instruction with the at least one processor of a computing device, using a fraction removal transformation, the first transformed representation of the mathematical operation to create a second transformed representation of the mathematical operation, the second transformed representation including a set of one or more polynomials that have one or more integer coefficients and are semantically equivalent to the first transformed representation; and
generating the computer readable instructions for parallel processing, the computer readable instructions generated based on the second transformed representation.
2. The method of claim 1 , wherein the computer language representation includes any one of:
matrix multiplication;
convolution;
max pooling; or
a derivative of a convolution.
3. The method of claim 1 , wherein the generation of the computer readable instructions from the second transformed representation of the mathematical operation takes less time computationally than generating computer readable instructions based on the computer language representation of the mathematical operation.
4. The method of claim 1 , wherein the computer language representation is of a machine learning linear algebra operation.
5. The method of claim 1 , wherein the first and second transformations of the computer language representation includes performing bound generation.
6. The method of claim 1 , wherein the computer readable instructions are executable on a computing device.
7. The method of claim 6 , wherein the generating of the computer readable instructions includes at least one of:
thread assignment;
memory layout; or
cache optimization.
8. A system to perform preprocessing of computer readable representations of mathematical operations, the system comprising:
a processor;
a memory including instructions that, when executed, cause the processor to at least:
perform an output reduction transformation to transform a computer readable representation of a machine learning mathematical operation into a first transformed representation including a set of one or more polynomials that are linear in a single variable, the first transformed representation semantically equivalent to the computer language representation;
perform a fraction removal transformation to transform the first transformed representation of a machine learning mathematical operation into a second transformed representation including a set of one or more polynomials that have one or more integer coefficients and semantically equivalent to the first transformed representation; and
generate computer readable instructions for parallel processing, the computer readable instructions generated based on the second transformed representation.
9. The system of claim 8 , wherein the processor is to generate the computer readable instructions in view of a target machine learning operation hardware accelerator that more optimally targets parallel processing.
10. The system of claim 8 , wherein the processor and memory are virtual and are provided via a hypervisor.
11. Computer storage media comprising instructions that, when executed, cause at least one processor to at least:
perform an output reduction transformation to transform source code including a linear algebra operation into a first transformed representation including a set of one or more polynomials that are linear in a single variable, the first transformed representation semantically equivalent to the source code;
perform a fraction removal transformation to transform the first transformed representation including a set of one or more polynomials that are linear in a single variable into a second transformed representation including a set of one or more polynomials that have one or more integer coefficients, the second transformed representation semantically equivalent to the first transformed representation; and
generate computer readable instructions for parallel processing, the computer readable instructions generated based on the second transformed representation.