Managing execution of a user defined function
The subject technology receives, in a computing process, a user defined function, the user defined function including code related to at least one operation to be performed. The subject technology determines by a security manager whether performing the at least one operation is permitted, the security manager determines restrictions, based at least in part on a security policy. The subject technology performs the at least one operation. The subject technology sends a result of the at least one operation to the computing process, where sending the result of the at least one operation utilizes a data transport mechanism that supports a network transfer of columnar data.
1. A system comprising:
at least one hardware processor; and
a memory storing instructions that cause the at least one hardware processor to perform operations comprising:
receiving, in a computing process, a user defined function, the user defined function including code related to at least one operation to be performed, wherein the computing process executes on an execution node, the computing process comprises an execution node process executing on the execution node, the execution node process executes a user defined function (UDF) client, and the UDF client communicates with a second computinq process and submits at least one request to perform a database query to the second computing process;
sending a request to perform the at least one operation on the database to a security manager engine;
determining by the security manager engine whether performing the at least one operation is permitted, the security manager engine determines restrictions, based at least in part on a security policy;
performing the at least one operation; and
sending a result of the at least one operation to the computing process, wherein sending the result of the at least one operation utilizes a data transport mechanism that supports a network transfer of columnar data.
2. The system of claim 1 , wherein the security manager engine forwards the at least one operation to a user code runtime executing within the security manager engine to perform the at least one operation.
3. The system of claim 1 , wherein the columnar data is transferred without memory copying or serialization.
4. The system of claim 1 , wherein the restrictions are on operations executing within a sandbox environment provided by the second computing process and the second computing process comprises a sandbox process for executing the at least one operation in the sandbox environment.
5. The system of claim 4 , wherein performing the at least one operation occurs within the sandbox environment.
6. The system of claim 5 , wherein the second computing process comprises a child process of the computing process, the computing process comprises an execution node process, the execution node process executing the user defined function client.
7. The system of claim 1 , wherein the computing process and the second computing process are executed on a same execution node of an execution platform.
8. The system of claim 7 , wherein the execution platform comprises a plurality of virtual warehouses, each virtual warehouse comprises a plurality of execution nodes, and a particular virtual warehouse from the plurality of virtual warehouse includes the same execution node that executes the computing process and the second computing process.
9. The system of claim 1 , wherein the UDF client is implemented in a programming language, and the code is related to at least one operation to be performed is implemented in a different programming language than the programming language.
10. A method comprising:
receiving, in a computing process, a user defined function, the user defined function including code related to at least one operation to be performed, wherein the computing process executes on an execution node, the computing process comprises an execution node process executing on the execution node, the execution node process executes a user defined function (UDF) client, and the UDF client communicates with a second computing process and submits at least one request to perform a database query to the second computing process;
sending a request to perform the at least one operation on the database to a security manager engine;
determining by the security manager engine whether performing the at least one operation is permitted, the security manager engine determines restrictions, based at least in part on a security policy;
performing the at least one operation; and
sending a result of the at least one operation to the computing process, wherein sending the result of the at least one operation utilizes a data transport mechanism that supports a network transfer of columnar data.
11. The method of claim 10 , wherein the security manager enqine forwards the at least one operation to a user code runtime executing within the security manager to perform the at least one operation.
12. The method of claim 10 , wherein the columnar data is transferred without memory copying or serialization.
13. The method of claim 10 , wherein the restrictions are on operations executing within a sandbox environment provided by the second computing process and the second computing process comprises a sandbox process for executing the at least one operation in the sandbox environment.
14. The method of claim 13 , wherein performing the at least one operation occurs within the sandbox environment.
15. The method of claim 14 , wherein the second computing process comprises a child process of the computing process, the computing process comprises an execution node process, the execution node process executing the user defined function client.
16. The method of claim 10 , wherein the computing process and the second computing process are executed on a same execution node of an execution platform.
17. The method of claim 16 , wherein the execution platform comprises a plurality of virtual warehouses, each virtual warehouse comprises a plurality of execution nodes, and a particular virtual warehouse from the plurality of virtual warehouse includes the same execution node that executes the computing process and the second computing process.
18. The method of claim 10 , wherein the UDF client is implemented in a programming language, and the code is related to at least one operation to be performed is implemented in a different programming language than the programming language.
19. A non-transitory computer-storage medium comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:
receiving, in a computing process, a user defined function, the user defined function including code related to at least one operation to be performed, wherein the computing process executes on an execution node, the computing process comprises an execution node process executing on the execution node, the execution node process executes a user defined function (UDF) client, and the UDF client communicates with a second computing process and submits at least one request to perform a database query to the second computing process;
sending a request to perform the at least one operation on the database to a security manager engine;
determining by the security manager engine whether performing the at least one operation is permitted, the security manager engine determines restrictions, based at least in part on a security policy;
performing the at least one operation; and sending a result of the at least one operation to the computing process, wherein sending the result of the at least one operation utilizes a data transport mechanism that supports a network transfer of columnar data.
20. The non-transitory computer-storage medium of claim 19 , wherein the security manager engine forwards the at least one operation to a user code runtime executing within the security manager engine to perform the at least one operation.
21. The non-transitory computer-storage medium of claim 19 , wherein the columnar data is transferred without memory copying or serialization.
22. The non-transitory computer-storage medium of claim 19 , wherein the restrictions are on operations executing within a sandbox environment provided by the second computing process and the second computing process comprises a sandbox process for executing the at least one operation in the sandbox environment.
23. The non-transitory computer-storage medium of claim 22 , wherein performing the at least one operation occurs within the sandbox environment.
24. The non-transitory computer-storage medium of claim 23 , wherein the second computing process comprises a child process of the computing process, the computing process comprises an execution node process, the execution node process executing user defined function client.
25. The non-transitory computer-storage medium of claim 19 , wherein the computing process and the second computing process are executed on a same execution node of an execution platform.
26. The non-transitory computer-storage medium of claim 25 , wherein the execution platform comprises a plurality of virtual warehouses, each virtual warehouse comprises a plurality of execution nodes, and a particular virtual warehouse from the plurality of virtual warehouse includes the same execution node that executes the computing process and the second computing process.
27. The non-transitory computer-storage medium of claim 19 , wherein the UDF client is implemented in a programming language, and the code is related to at least one operation to be performed is implemented in a different programming language than the programming language.