IP Library › Granted Patent US 11,487,778
Granted Patent B1
US 11,487,778 · App. 17/649,571 · Granted Nov 1, 2022

Managed event tables in a database system

Inventors: Benoit Dageville (San Mateo, CA); Ganeshan Ramachandran Iyer (Redmond, WA); Unmesh Jagtap (San Mateo, CA); Subramanian Muralidhar (Mercer Island, WA)
Assignee: Snowflake Inc.
G06F16/254G06F16/2282G06F16/258
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,487,778
App. No.
17/649,571
Filed
Feb 1, 2022
Granted
Nov 1, 2022
Kind
B1
Examiner
VY, HUNG T
Art Unit
2163
USPC
707/602
Abstract

Provided herein are systems and methods for configuring managed event tables. A system includes at least one hardware processor coupled to a memory and configured to collect using an event table instance within a sandbox process, log data generated by a UDF during execution of the UDF code. The log data is provided from the sandbox process to an execution node process of the execution node. The log data is formatted using the execution node process, to generate formatted log data. The formatting is based on a configuration of a managed event table that is external to the execution node. The at least one hardware processor further causes ingestion of the formatted log data from the execution node process into the managed event table.

Claims (87)

1. A system comprising:

at least one hardware processor; and

at least one memory storing instructions that cause the at least one hardware processor to perform operations comprising:

instantiating a user code runtime to execute code of a user-defined function (UDF), the user code runtime instantiated within a sandbox process of an execution node, the sandbox process configured to apply a system call filter to a process of the user code runtime associated with the UDF, while operating in a secure computing mode;

collecting using an event table instance within the sandbox process, log data generated by the UDF during execution of the code;

providing the log data from the sandbox process to an execution node process of the execution node;

formatting the log data using the execution node process, to generate formatted log data, the formatting based on a configuration of a managed event table that is external to the execution node; and

causing ingestion of the formatted log data from the execution node process into the managed event table.

2. The system of claim 1 , wherein the operations further comprise:

configuring the sandbox process with managed event table libraries, the managed event table libraries causing execution of the event table instance.

3. The system of claim 1 , wherein the operations further comprise:

enriching at the execution node process, the log data with contextual metadata describing the log data, yielding enriched log data.

4. The system of claim 3 , wherein the contextual metadata comprises at least one of:

a timestamp identifying a time the log data was generated;

a query identifier of a query causing the generation of the log data;

a session identifier of a session during which the log data was generated; and

a warehouse identifier of a virtual warehouse of computing resources that include the execution node.

5. The system of claim 3 , wherein the operations further comprise:

storing the enriched log data at a table stage storage maintained by the execution node process.

6. The system of claim 5 , wherein the configuration comprises a format that is ingestible by the managed event table.

7. The system of claim 6 , wherein the operations further comprise:

formatting the enriched log data stored in the table stage storage into the format that is ingestible by the managed event table, yielding the formatted log data.

8. The system of claim 7 , wherein the operations further comprise:

scheduling the ingestion of the formatted log data into the managed event table, based on an indication the formatted log data is available at the table stage storage.

9. The system of claim 1 , wherein the operations further comprise:

decoding a query for the log data generated by the UDF during execution of the code;

executing the query using the formatted log data stored in the managed event table to generate a query result; and

outputting the query result in response to the query.

10. The system of claim 1 , wherein the operations further comprise:

collecting the log data further based on a data access privilege indicated by the configuration of the managed event table.

11. A method comprising:

performing by at least one hardware processor:

instantiating within a sandbox process of an execution node, a user code runtime to execute code of a user-defined function (UDF), the sandbox process configured to apply a system call filter to a process of the user code runtime associated with the UDF, while operating in a secure computing mode;

collecting using an event table instance within the sandbox process, log data generated by the UDF during execution of the code;

providing the log data from the sandbox process to an execution node process of the execution node;

formatting of the log data using the execution node process, to generate formatted log data, the formatting based on a configuration of a managed event table that is external to the execution node; and

causing ingestion of the formatted log data from the execution node process into the managed event table.

12. The method of claim 11 , further comprising:

configuring the sandbox process with managed event table libraries, the managed event table libraries causing execution of the event table instance.

13. The method of claim 11 , further comprising:

enriching at the execution node process, the log data with contextual metadata describing the log data, yielding enriched log data.

14. The method of claim 13 , wherein the contextual metadata comprises at least one of:

a timestamp identifying a time the log data was generated;

a query identifier of a query causing the generation of the log data;

a session identifier of a session during which the log data was generated; and

a warehouse identifier of a virtual warehouse of computing resources that include the execution node.

15. The method of claim 13 , further comprising:

storing the enriched log data at a table stage storage maintained by the execution node process.

16. The method of claim 15 , wherein the configuration comprises a format that is ingestible by the managed event table.

17. The method of claim 16 , further comprising:

formatting the enriched log data stored in the table stage storage into the format that is ingestible by the managed event table, yielding the formatted log data.

18. The method of claim 17 , further comprising:

scheduling the ingestion of the formatted log data into the managed event table, based on an indication the formatted log data is available at the table stage storage.

19. The method of claim 11 , further comprising:

decoding a query for the log data generated by the UDF during execution of the code;

executing the query using the formatted log data stored in the managed event table to generate a query result; and

outputting the query result in response to the query.

20. The method of claim 11 , further comprising:

collecting the log data further based on a data access privilege indicated by the configuration of the managed event table.

21. A computer-readable medium comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:

instantiating within a sandbox process of an execution node, a user code runtime to execute code of a user-defined function (UDF), the sandbox process configured to apply a system call filter to a process of the user code runtime associated with the UDF, while operating in a secure computing mode;

collecting using an event table instance within the sandbox process, log data generated by the UDF during execution of the code;

providing the log data from the sandbox process to an execution node process of the execution node;

formatting of the log data using the execution node process, to generate formatted log data, the formatting based on a configuration of a managed event table that is external to the execution node; and

causing ingestion of the formatted log data from the execution node process into the managed event table.

22. The computer-readable medium of claim 21 , the operations further comprising:

configuring the sandbox process with managed event table libraries, the managed event table libraries causing execution of the event table instance.

23. The computer-readable medium of claim 21 , the operations further comprising:

enriching at the execution node process, the log data with contextual metadata describing the log data, yielding enriched log data.

24. The computer-readable medium of claim 23 , wherein the contextual metadata comprises at least one of:

a timestamp identifying a time the log data was generated;

a query identifier of a query causing the generation of the log data;

a session identifier of a session during which the log data was generated; and

a warehouse identifier of a virtual warehouse of computing resources that include the execution node.

25. The computer-readable medium of claim 23 , further comprising:

storing the enriched log data at a table stage storage maintained by the execution node process.

26. The computer-readable medium of claim 25 , wherein the configuration comprises a format that is ingestible by the managed event table.

27. The computer-readable medium of claim 26 , the operations further comprising:

formatting the enriched log data stored in the table stage storage into the format that is ingestible by the managed event table, yielding the formatted log data.

28. The computer-readable medium of claim 27 , the operations further comprising:

scheduling the ingestion of the formatted log data into the managed event table, based on an indication the formatted log data is available at the table stage storage.

29. The computer-readable medium of claim 21 , the operations further comprising:

decoding a query for the log data generated by the UDF during execution of the code;

executing the query using the formatted log data stored in the managed event table to generate a query result; and

outputting the query result in response to the query.

30. The computer-readable medium of claim 21 , the operations further comprising:

collecting the log data further based on a data access privilege indicated by the configuration of the managed event table.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2022
From: DAGEVILLE, BENOIT; IYER, GANESHAN RAMACHANDRAN; JAGTAP, UNMESH; MURALIDHAR, SUBRAMANIAN
To: SNOWFLAKE INC.
Reel/Frame 059090/0252 →
Continuity (1)
Provisional Application 63263737 · Nov 8, 2021
Cited By (4)
US 12,210,494 US 12,517,897 US 12,717,805 US 12,724,761