Accessing data using a file reference-based user defined function
A method includes decoding, by at least one hardware processor, a request for a user-defined function (UDF). The request includes a reference to one or more files. The method further includes generating, by the at least one hardware processor, the UDF based on the request. The UDF includes a file reference object with file path information corresponding to the reference. The file path information identifies a file path to the one or more files. A UDF call into the UDF is detected. The UDF call specifies the file path information. The UDF call is processed to generate result data using the one or more files.
1. A method comprising:
decoding, by at least one hardware processor, a request for a user-defined function (UDF), the request including a reference to one or more files;
generating in a UDF execution environment, by the at least one hardware processor, the UDF based on the request, the UDF including a file reference object with file path information corresponding to the reference, and the file path information identifying a file path to the one or more files specified by the request;
detecting a UDF call into the UDF, the UDF call specifying the file path information of the file reference object; and
processing the UDF call in a sandbox process to access the one or more files via the file path and to generate result data using the one or more files.
2. The method of claim 1 , further comprising:
parsing the file reference object to determine a storage location storing the one or more files.
3. The method of claim 2 , further comprising:
retrieving the one or more files from the storage location using the file path;
applying a function associated with the UDF to the one or more files to generate the result data; and
storing the result data.
4. The method of claim 1 , further comprising:
configuring the file reference object to pass the file path to a class of the UDF.
5. The method of claim 4 , further comprising:
detecting the UDF call specifies the file path and includes a call to the class of the UDF; and
processing the UDF call using the class of the UDF, the processing including retrieving the one or more files from a storage location indicated by the file path.
6. The method of claim 5 , wherein the class of the UDF indicates a database type associated with the storage location, and the method further comprising:
processing the UDF call based on the database type associated with the storage location.
7. The method of claim 1 , further comprising:
detecting the UDF call at a function server of a network-based database system.
8. The method of claim 7 , further comprising:
transmitting the UDF call to a network node using a remote procedure call (RPC) channel, the network node being separate from the function server.
9. The method of claim 8 , further comprising:
processing the UDF in a runtime environment executing on the network node.
10. The method of claim 9 , further comprising:
executing the runtime environment within a sandbox process configured as a sub-process of the network node.
11. A system comprising:
one or more processors of a machine; and
at least one memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
decoding a request for a user-defined function (UDF), the request including a reference to one or more files;
generating in a UDF execution environment, the UDF based on the request, the UDF including a file reference object with file path information corresponding to the reference, and the file path information identifying a file path to the one or more files specified by the request;
detecting a UDF call into the UDF, the UDF call specifying the file path information of the file reference object; and
processing the UDF call in a sandbox process to access the one or more files via the file path and to generate result data using the one or more files.
12. The system of claim 11 , the operations further comprising:
parsing the file reference object to determine a storage location storing the one or more files.
13. The system of claim 12 , further comprising:
retrieving the one or more files from the storage location using the file path;
applying a function associated with the UDF to the one or more files to generate the result data; and
storing the result data.
14. The system of claim 11 , the operations further comprising:
configuring the file reference object to pass the file path to a class of the UDF.
15. The system of claim 14 , the operations further comprising:
detecting the UDF call specifies the file path and includes a call to the class of the UDF; and
processing the UDF call using the class of the UDF, the processing including retrieving the one or more files from a storage location indicated by the file path.
16. The system of claim 15 , wherein the class of the UDF indicates a database type associated with the storage location, and the operations further comprising:
processing the UDF call based on the database type associated with the storage location.
17. The system of claim 11 , the operations further comprising:
detecting the UDF call at a function server of a network-based database system.
18. The system of claim 17 , the operations further comprising:
transmitting the UDF call to a network node using a remote procedure call (RPC) channel, the network node being separate from the function server.
19. The system of claim 18 , the operations further comprising:
processing the UDF in a runtime environment executing on the network node.
20. The system of claim 19 , the operations further comprising:
executing the runtime environment within a sandbox process configured as a sub-process of the network node.
21. A machine storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
decoding a request for a user-defined function (UDF), the request including a reference to one or more files;
generating in a UDF execution environment, the UDF based on the request, the UDF including a file reference object with file path information corresponding to the reference, and the file path information identifying a file path to the one or more files specified by the request;
detecting a UDF call into the UDF, the UDF call specifying the file path information of the file reference object; and
processing the UDF call in a sandbox process to access the one or more files via the file path and to generate result data using the one or more files.
22. The machine storage medium of claim 21 , the operations further comprising:
parsing the file reference object to determine a storage location storing the one or more files.
23. The machine storage medium of claim 22 , further comprising:
retrieving the one or more files from the storage location using the file path;
applying a function associated with the UDF to the one or more files to generate the result data; and
storing the result data.
24. The machine storage medium of claim 21 , the operations further comprising:
configuring the file reference object to pass the file path to a class of the UDF.
25. The machine storage medium of claim 24 , the operations further comprising:
detecting the UDF call specifies the file path and includes a call to the class of the UDF; and
processing the UDF call using the class of the UDF, the processing including retrieving the one or more files from a storage location indicated by the file path.
26. The machine storage medium of claim 25 , wherein the class of the UDF indicates a database type associated with the storage location, and the operations further comprising:
processing the UDF call based on the database type associated with the storage location.
27. The machine storage medium of claim 21 , the operations further comprising:
detecting the UDF call at a function server of a network-based database system.
28. The machine storage medium of claim 27 , the operations further comprising:
transmitting the UDF call to a network node using a remote procedure call (RPC) channel, the network node being separate from the function server.
29. The machine storage medium of claim 28 , the operations further comprising:
processing the UDF in a runtime environment executing on the network node.
30. The machine storage medium of claim 29 , the operations further comprising:
executing the runtime environment within a sandbox process configured as a sub-process of the network node.