Executing commands from a distributed execution model
Systems and methods are disclosed for generating a distributed execution model with untrusted commands. The system can receive a query, and process the query to identify the untrusted commands. The system can use data associated with the untrusted command to identify one or more files associated with the untrusted command. Based on the files, the system can generate a data structure and include one or more identifiers associated with the data structure in the distributed execution model. The system can distribute the distributed execution model to one or more nodes in a distributed computing environment for execution.
1. A computer-implemented method, comprising:
identifying, by a first computing system, a command associated with a second computing system from a plurality of commands of a distributed execution model to execute at least a portion of a query;
generating a data structure in a restricted computing environment based on the command; and
executing one or more computer executable instructions based on the data structure.
2. The computer-implemented method of claim 1 , wherein the command comprises an untrusted command.
3. The computer-implemented method of claim 1 , wherein a command associated with the first computing system comprises a trusted command.
4. The computer-implemented method of claim 1 , further comprising receiving the distributed execution model.
5. The computer-implemented method of claim 1 , wherein the plurality of commands correspond to a plurality of commands of the query.
6. The computer-implemented method of claim 1 , further comprising adding an identifier associated with the data structure to the distributed execution model.
7. The computer-implemented method of claim 1 , wherein identifying the command comprises identifying the command based on a syntax model.
8. The computer-implemented method of claim 7 , wherein the syntax model comprises an abstract syntax tree.
9. The computer-implemented method of claim 7 , wherein the syntax model comprises a plurality of command nodes corresponding to a plurality of commands of the query.
10. The computer-implemented method of claim 7 , wherein the syntax model identifies one or more trusted commands and one or more untrusted commands.
11. The computer-implemented method of claim 1 , wherein the distributed execution model comprises a distributed acyclic graph.
12. The computer-implemented method of claim 1 , wherein the command is associated with libraries and dependencies that are not known by a system executing the at least a portion of the query.
13. The computer-implemented method of claim 1 , wherein each of the plurality of commands is associated with one or more computation operations.
14. The computer-implemented method of claim 1 , wherein the plurality of commands include a trusted command that is associated with libraries and dependencies that are known by a system executing the at least a portion of the query.
15. The computer-implemented method of claim 1 , further comprising instantiating the restricted computing environment to generate the data structure.
16. The computer-implemented method of claim 1 , further comprising communicating data associated with the command to the restricted computing environment to generate the data structure.
17. The computer-implemented method of claim 1 , wherein generating the data structure comprises:
communicating data associated with the command to the restricted computing environment using an external transformer; and
receiving the data structure from the restricted computing environment using the external transformer.
18. The computer-implemented method of claim 1 , wherein generating the data structure comprises communicating data associated with the command to the restricted computing environment, wherein the data associated with the command includes a filename and a file location associated with a file, wherein the restricted computing environment uses the filename and the file location to identify the file and uses the file to generate the data structure.
19. A first computing system, comprising:
memory; and
one or more processing devices coupled to the memory and configured to:
identify, a command associated with a second computing system from a plurality of commands of a distributed execution model to execute at least a portion of a query;
generate a data structure in a restricted computing environment based on the command; and
execute one or more computer executable instructions based on the data structure.
20. A non-transitory computer readable media comprising computer-executable instructions that, when executed by a first computing system, cause the first computing system to:
identify, a command associated with a second computing system from a plurality of commands of a distributed execution model to execute at least a portion of a query;
generate a data structure in a restricted computing environment based on the command; and
execute one or more computer executable instructions based on the data structure.