Extended telemetry metric data types
Provided herein are systems and methods for configuring extended telemetry metric data types. A system includes at least one hardware processor coupled to a memory and configured to detect an API call at an execution node of a database system. Metric data emitted by a telemetry API corresponding to the API call is collected. The metric data is converted from a first data format associated with the telemetry API to a second data format to generate formatted metric data. The second data format is a native format to an event table of the database system. Telemetry data based on the formatted metric data is ingested into the event table.
1 . A system comprising:
at least one hardware processor; and
at least one memory storing instructions that cause the at least one hardware processor to perform operations comprising:
detecting an application programming interface (API) call at an execution node of a database system;
collecting metric data emitted by a telemetry API, the telemetry API being pre-mapped to the API call within a processing stack of an execution node process configured at the execution node;
converting the metric data from a first data format associated with the telemetry API to a second data format to generate formatted metric data, the second data format being native to an event table of the database system; and
causing ingestion of telemetry data into the event table based on an execution of a command during the execution node process, the telemetry data based on the formatted metric data.
2 . The system of claim 1 , the operations further comprising:
executing a COPY command to cause the ingestion of the telemetry data into the event table.
3 . The system of claim 2 , the operations further comprising:
transforming the formatted metric data into the telemetry data during the executing of the COPY command, the transforming based on at least one metric data type of a plurality of metric data types associated with the metric data.
4 . The system of claim 3 , the operations further comprising:
matching during the ingestion, a value type field in the formatted metric data with a corresponding value type column in the event table, wherein the value type field indicates a metric data type associated with the metric data.
5 . The system of claim 3 , the operations further comprising:
configuring the transforming of the formatted metric data during the executing of the COPY command using a data manipulation manager in an execution node process of the execution node.
6 . The system of claim 3 , the operations further comprising:
detecting availability of the formatted metric data in a table stage associated with an execution node process of the execution node; and
scheduling the ingestion based on the detecting of the availability of the formatted metric data.
7 . The system of claim 6 , the operations further comprising:
detecting a completion of the scheduling; and
executing the COPY command based on detection of the completion of the scheduling.
8 . The system of claim 1 , wherein the first data format is an OpenTelemetry (OTEL) JavaScript Object Notation (JSON) format, and the second data format is an event table JSON format.
9 . The system of claim 1 , wherein the metric data is associated with a metric data type of a plurality of metric data types, and wherein the plurality of metric data types comprises:
an integer metric data type;
a double metric data type;
a Boolean metric data type;
a string metric data type;
a binary metric data type;
an array metric data type;
an object metric data type;
a timestamp metric data type; and
a null value metric data type.
10 . The system of claim 1 , the operations further comprising:
instantiating a user code runtime to execute user-defined function (UDF) code, the user code runtime instantiated within a sandbox process of the execution node; and
detecting the API call during the execution of the UDF code.
11 . A method comprising:
detecting, by at least one hardware processor, an application programming interface (API) call at an execution node of a database system;
collecting metric data emitted by a telemetry API, the telemetry API being pre-mapped to the API call within a processing stack of an execution node process configured at the execution node;
converting the metric data from a first data format associated with the telemetry API to a second data format to generate formatted metric data, the second data format being native to an event table of the database system; and
causing ingestion of telemetry data into the event table based on an execution of a command during the execution node process, the telemetry data based on the formatted metric data.
12 . The method of claim 11 , further comprising:
executing a COPY command to cause the ingestion of the telemetry data into the event table.
13 . The method of claim 12 , further comprising:
transforming the formatted metric data into the telemetry data during the executing of the COPY command, the transforming based on at least one metric data type of a plurality of metric data types associated with the metric data.
14 . The method of claim 13 , further comprising:
matching during the ingestion, a value type field in the formatted metric data with a corresponding value type column in the event table, wherein the value type field indicates a metric data type associated with the metric data.
15 . The method of claim 13 , further comprising:
configuring the transforming of the formatted metric data during the executing of the COPY command using a data manipulation manager in an execution node process of the execution node.
16 . The method of claim 13 , further comprising:
detecting availability of the formatted metric data in a table stage associated with an execution node process of the execution node; and
scheduling the ingestion based on the detecting of the availability of the formatted metric data.
17 . The method of claim 16 , further comprising:
detecting a completion of the scheduling; and
executing the COPY command based on detection of the completion of the scheduling.
18 . The method of claim 11 , wherein the first data format is an OpenTelemetry (OTEL) JavaScript Object Notation (JSON) format, and the second data format is an event table JSON format.
19 . The method of claim 11 , wherein the metric data is associated with a metric data type of a plurality of metric data types, and wherein the plurality of metric data types comprises:
an integer metric data type;
a double metric data type;
a Boolean metric data type;
a string metric data type;
a binary metric data type;
an array metric data type;
an object metric data type;
a timestamp metric data type; and
a null value metric data type.
20 . The method of claim 11 , further comprising:
instantiating a user code runtime to execute user-defined function (UDF) code, the user code runtime instantiated within a sandbox process of the execution node; and
detecting the API call during the execution of the UDF code.
21 . A computer-storage medium comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:
detecting an application programming interface (API) call at an execution node of a database system;
collecting metric data emitted by a telemetry API, the telemetry API being pre-mapped to the API call within a processing stack of an execution node process configured at the execution node;
converting the metric data from a first data format associated with the telemetry API to a second data format to generate formatted metric data, the second data format being native to an event table of the database system; and
causing ingestion of telemetry data into the event table based on an execution of a command during the execution node process, the telemetry data based on the formatted metric data.
22 . The computer-storage medium of claim 21 , the operations further comprising:
executing a COPY command to cause the ingestion of the telemetry data into the event table.
23 . The computer-storage medium of claim 22 , the operations further comprising:
transforming the formatted metric data into the telemetry data during the executing of the COPY command, the transforming based on at least one metric data type of a plurality of metric data types associated with the metric data.
24 . The computer-storage medium of claim 23 , the operations further comprising:
matching during the ingestion, a value type field in the formatted metric data with a corresponding value type column in the event table, wherein the value type field indicates a metric data type associated with the metric data.
25 . The computer-storage medium of claim 23 , the operations further comprising:
configuring the transforming of the formatted metric data during the executing of the COPY command using a data manipulation manager in an execution node process of the execution node.
26 . The computer-storage medium of claim 23 , the operations further comprising:
detecting availability of the formatted metric data in a table stage associated with an execution node process of the execution node; and
scheduling the ingestion based on the detecting of the availability of the formatted metric data.
27 . The computer-storage medium of claim 26 , the operations further comprising:
detecting a completion of the scheduling; and
executing the COPY command based on detection of the completion of the scheduling.
28 . The computer-storage medium of claim 21 , wherein the first data format is an Open Telemetry (OTEL) JavaScript Object Notation (JSON) format, and the second data format is an event table JSON format.
29 . The computer-storage medium of claim 21 , wherein the metric data is associated with a metric data type of a plurality of metric data types, and wherein the plurality of metric data types comprises:
an integer metric data type;
a double metric data type;
a Boolean metric data type;
a string metric data type;
a binary metric data type;
an array metric data type;
an object metric data type;
a timestamp metric data type; and
a null value metric data type.
30 . The computer-storage medium of claim 21 , the operations further comprising:
instantiating a user code runtime to execute user-defined function (UDF) code, the user code runtime instantiated within a sandbox process of the execution node; and
detecting the API call during the execution of the UDF code.