Automated expression parallelization
A system is capable of automatically adjusting or reconstructing a baseline expression to generate a parallelized expression. Evaluation of the parallelized expression provide a substantially similar output as the evaluation of the baseline query in more efficient manner. In some implementations, data indicating an expression to be evaluated on a primary thread of the one or more processors is obtained. Elements of the expression are identified. The elements are grouped into a parse tree representation. Elements of the expression are classified as belonging to either a first category that includes elements that are eligible for parallel processing or a second category that includes elements that are not eligible for parallel processing. A particular element that is classified as belonging to the first category is identified and evaluated on a non-primary thread of the one or more processors. The non-primary thread is evaluated in parallel with the primary thread.
1. A method comprising:
obtaining data indicating a baseline expression comprising at least a first element and a second element;
determining that evaluation of the second element within the baseline expression involves accessing at least one external resource;
determining that a predicted time to evaluate the second element based on accessing the at least one external resource satisfies a predetermined threshold;
based on (i) determining that evaluation of the second element within the baseline expression involves accessing the at least one external resource and (ii) determining that the predicted time to evaluate the second element satisfies the predetermined threshold, determining that the second element is eligible for parallel processing;
generating a transformed expression corresponding to the baseline expression based on determining that the second element is eligible for parallel processing,
wherein the transformed expression comprises instructions that, when the transformed expression is evaluated by one or more processors, cause (i) the first element and the second element to each be evaluated on a first processor thread of the one or more processors, (ii) other computer-executable elements of the baseline expression to be evaluated on a second processor thread of the one or more processors, and (iii) the first processor thread to be executed in parallel with the second processor thread; and
providing the transformed expression for output.
2. The method of claim 1 , wherein:
the transformed expression is coded in a declarative programming language; and
the first element, the second element, and the other computer-executable elements are each associated with an imperative programming language executed by the one or more processors.
3. The method of claim 1 , wherein:
the first processor thread comprises a primary thread of the one or more processors; and
the second processor thread comprises a non-primary thread of the one or more processors.
4. The method of claim 1 , wherein the instructions cause the first element to be evaluated on the first processor thread prior to evaluation of the second element on the first processor thread.
5. The method of claim 1 , wherein generating the transformed expression corresponding to the baseline expression comprises reconstructing a first text segment of the baseline expression to generate a second text segment for the transformed expression.
6. The method of claim 1 , wherein generating the transformed expression corresponding to the baseline expression comprises generating a parse tree representation for the baseline expression, wherein the parse tree representation comprises a hierarchical structure of nodes that identifies computation dependencies of computer-executable elements of the baseline expression.
7. The method of claim 6 , wherein generating the transformed expression corresponding to the baseline expression comprises:
determining a particular sequence to evaluate the computer-executable elements of the baseline expression; and
based on determining the particular sequence:
assigning the first element to a first child node of an intermediate note within the parse tree representation, and
assigning the second element to a second child node of the intermediate note within the parse tree representation.
8. The method of claim 1 , wherein:
the baseline expression is coded in a declarative programming language developed using an application development platform; and
determining that evaluation of the second element within the baseline expression involves accessing at least one external resource by determining that evaluation of the second element within the baseline expression involves accessing a resource external to the application development platform.
9. The method of claim 8 , wherein:
the resource external to the application development platform comprises a remote web service operating outside of the application development platform; and
determining that evaluation of the second element within the baseline expression involves accessing the resource external to the application development platform by determining that evaluation of the second element within the baseline expression involves providing a request message to the remote web service.
10. A system comprising:
one or more computing devices; and
a non-transitory computer-readable storage device coupled to the one or more computing devices having instructions stored thereon, which, when executed by the one or more computing devices, cause the one or more computing devices to perform operations comprising:
obtaining data indicating a baseline expression comprising at least a first element and a second element;
determining that evaluation of the second element within the baseline expression involves accessing at least one external resource;
determining that a predicted time to evaluate the second element based on accessing the at least one external resource satisfies a predetermined threshold;
based on (i) determining that evaluation of the second element within the baseline expression involves accessing the at least one external resource and (ii) determining that the predicted time to evaluate the second element satisfies the predetermined threshold, determining that the second element is eligible for parallel processing;
generating a transformed expression corresponding to the baseline expression based on determining that the second element is eligible for parallel processing,
wherein the transformed expression comprises instructions that, when the transformed expression is evaluated by one or more processors, cause (i) the first element and the second element to each be evaluated on a first processor thread of the one or more processors, (ii) other computer-executable elements of the baseline expression to be evaluated on a second processor thread of the one or more processors, and (iii) the first processor thread to be executed in parallel with the second processor thread; and
providing the transformed expression for output.
11. The system of claim 10 , wherein:
the transformed expression is coded in a declarative programming language; and
the first element, the second element, and the other computer-executable elements are each associated with an imperative programming language executed by the one or more processors.
12. The system of claim 10 , wherein:
the first processor thread comprises a primary thread of the one or more processors; and
the second processor thread comprises a non-primary thread of the one or more processors.
13. The system of claim 10 , wherein the instructions cause the first element to be evaluated on the first processor thread prior to evaluation of the second element on the first processor thread.
14. The system of claim 10 , wherein generating the transformed expression corresponding to the baseline expression comprises generating a parse tree representation for the baseline expression, wherein the parse tree representation comprises a hierarchical structure of nodes that identifies computation dependencies of computer-executable elements of the baseline expression.
15. The system of claim 14 , wherein generating the transformed expression corresponding to the baseline expression comprises:
determining a particular sequence to evaluate the computer-executable elements of the baseline expression; and
based on determining the particular sequence:
assigning the first element to a first child node of an intermediate note within the parse tree representation, and
assigning the second element to a second child node of the intermediate note within the parse tree representation.
16. A non-transitory computer-readable storage device having instructions stored thereon, which, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:
obtaining data indicating a baseline expression comprising at least a first element and a second element;
determining that evaluation of the second element within the baseline expression involves accessing at least one external resource;
determining that a predicted time to evaluate the second element based on accessing the at least one external resource satisfies a predetermined threshold;
based on (i) determining that evaluation of the second element within the baseline expression involves accessing the at least one external resource and (ii) determining that the predicted time to evaluate the second element satisfies the predetermined threshold, determining that the second element is eligible for parallel processing;
generating a transformed expression corresponding to the baseline expression based on determining that the second element is eligible for parallel processing,
wherein the transformed expression comprises instructions that, when the transformed expression is evaluated by one or more processors, cause (i) the first element and the second element to each be evaluated on a first processor thread of the one or more processors, (ii) other computer-executable elements of the baseline expression to be evaluated on a second processor thread of the one or more processors, and (iii) the first processor thread to be executed in parallel with the second processor thread; and
providing the transformed expression for output.
17. The non-transitory computer-readable storage device of claim 16 , wherein:
the transformed expression is coded in a declarative programming language; and
the first element, the second element, and the other computer-executable elements are each associated with an imperative programming language executed by the one or more processors.
18. The non-transitory computer-readable storage device of claim 16 , wherein:
the first processor thread comprises a primary thread of the one or more processors; and
the second processor thread comprises a non-primary thread of the one or more processors.
19. The non-transitory computer-readable storage device of claim 16 , wherein the instructions cause the first element to be evaluated on the first processor thread prior to evaluation of the second element on the first processor thread.
20. The non-transitory computer-readable storage device of claim 16 , wherein generating the transformed expression corresponding to the baseline expression comprises generating a parse tree representation for the baseline expression, wherein the parse tree representation comprises a hierarchical structure of nodes that identifies computation dependencies of computer-executable elements of the baseline expression.