Click-to-script reflection
A click-to-script service enables developers of big-data job scripts to quickly see the underlying script operations from optimized execution plans. Once a big-data job is received, the disclosed examples compile it and generate tokens that are associated with each operation of the big-data job. These tokens include may include the file name of the job, the line number of the operation, and/or an Abstract Syntax Tree (AST) node for the given operations. An original execution plan is optimized into an optimized execution plan, and the tokens for the original operations of the job script are assigned to the optimized operations of the optimized execution plan. The optimized execution plan is graphically displayed in an interactive manner such that users may view the optimized execution plan and click on its optimized operations to find the original operations of the job script.
1 . A computerized method comprising:
generating a token for a job script operation of a job, the token comprising:
a file name of a job script, and
a position of an Abstract Syntax Tree (AST) node for an original execution plan of the job;
assigning the generated token with a different operation of an optimized execution plan of the job script; and
providing a graphical representation of the different operation as a leaf of an AST of the optimized execution plan in a user interface, wherein the graphical representation allows a user to navigate from the leaf representing the different operation to the job script operation.
2 . The method of claim 1 , further comprising:
wherein interdependencies of the leaf are visually shown in the graphical representation.
3 . The method of claim 1 , further comprising:
visualizing a job execution graph of the optimized execution plan.
4 . The method of claim 1 , further comprising:
generating an equivalent operator tree to optimize the original execution plan into the optimized execution plan.
5 . The method of claim 4 , wherein generating the equivalent operator tree further comprises:
copying an original query of the job script into an internal memo structure; and
initiating a task to optimize a class corresponding to a root node of a query tree of the original execution plan.
6 . The method of claim 5 , wherein generating the equivalent operator tree further comprises:
in response to initiating the task to optimize the class corresponding to the root node, initiating a task to optimize a subtree of the query tree.
7 . The method of claim 4 , wherein generating the equivalent operator tree further comprises:
performing an optimization task according to a Volcano technique.
8 . The method of claim 7 , wherein generating the equivalent operator tree further comprises:
checking whether an optimization goal has already been pursued prior to performing the optimization task.
9 . The method of claim 4 , wherein generating the equivalent operator tree further comprises:
performing an optimization task according to a Cascades technique.
10 . The method of claim 9 , wherein generating the equivalent operator tree further comprises:
excluding an optimization rule based on taking a transitive closure of a reachability relationship of operators mapped to each other in a single rule application.
11 . A computerized method comprising:
applying a token to a job script operation of a job script for a job, wherein the token comprises a position of an Abstract Syntax Tree (AST) node for an original execution plan of the job;
optimizing the original execution plan to create an optimized execution plan of the job, wherein the optimized execution plan comprises a different operation than the job script operation;
associating the token applied to the job script operation with the different operation of the optimized execution plan; and
providing a graphical representation of the different operation as a leaf of an AST of the optimized execution plan in a user interface, wherein the graphical representation allows a user to navigate from the leaf representing the different operation to the job script operation.
12 . The method of claim 11 , further comprising:
optimizing the original execution plan into the optimized execution plan through a Volcano search strategy.
13 . The method of claim 12 , further comprising:
checking whether an optimization goal has already been pursued prior to optimizing the original execution plan into the optimized execution plan.
14 . The method of claim 11 , further comprising:
optimizing the original execution plan into the optimized execution plan through a Cascades optimizer.
15 . The method of claim 14 , further comprising:
excluding an optimization rule based on taking a transitive closure of a reachability relationship of operators mapped to each other in a single rule application.
16 . A system comprising:
a memory embodied with instructions to add a token to a job script operation of a job and optimize the job script operations with the added token into an optimized execution plan; and
a processor configured to:
apply a token to a job script operation, wherein the token comprises a position of an Abstract Syntax Tree (AST) node for an original execution plan of the job;
optimize the original execution plan into the optimized execution plan of the job, wherein the optimized execution plan comprises a different operation than the job script operation;
associate the token applied to the job script operation with the different operation of the optimized execution plan; and
provide a graphical representation of the different operation as a leaf of an AST of the optimized execution plan to a user interface of a client computing device, wherein the graphical representation of the different operation is interactive and links from the leaf representing the different operation to the job script operation.
17 . The system of claim 16 , wherein the processor is further configured to:
optimize the original execution plan into the optimized execution plan through a Volcano search strategy.
18 . The system of claim 17 , wherein the processor is further configured to:
check whether an optimization goal has already been pursued prior to optimizing the original execution plan into the optimized execution plan.
19 . The system of claim 16 , wherein the processor is further configured to:
optimize the original execution plan into the optimized execution plan through a Cascades optimizer.
20 . The system of claim 19 , wherein the processor is further configured to:
exclude an optimization rule based on taking a transitive closure of a reachability relationship of operators mapped to each other in a single rule application.