Data query method, data query apparatus, and computer-program product
A data query method is provided. The data query method includes receiving query description; processing, by a processor, the query description using one or more components; and determining query statement. Processing the query description includes obtaining query field from the query description; processing the query field using the one or more components; and determining a first sub-statement.
1 . A data query method, comprising:
receiving query description that specifies at least one data source type and at least one query parameter;
loading an interface of at least one user-defined component onto a memory, obtaining a corresponding implementation of the at least one user-defined component, and grouping obtained implementations based on the at least one data source type;
processing, by a processor, the query description using one or more components, by identifying the at least one data source type and executing data-source-specific operations associated with the at least one data source type to generate query statement that conforms to specified query parameters; and
dynamically adjusting the query statement based on the at least one data source type to enhance compatibility and processing efficiency;
wherein processing the query description comprises:
obtaining query field from the query description;
processing the query field using the one or more components;
determining a first sub-statement;
obtaining filter type field from the query description that defines a specific subset of data types within the data source;
processing the filter type field by loading an interface of a dedicated filtering component onto the memory, obtaining its implementation, and processing one or more dedicated filtering components configured to handle character, numeric, or date data types to generate a filter sub-statement that reflects the filtered results in a format compatible with the specified data source; and
determining a query statement by joining the first sub-statement with the filter sub-statement.
2 . The data query method of claim 1 , wherein processing the query description comprises:
determining whether the query field comprises an aggregate query field; and
processing the aggregate query field using a first component;
wherein the first component is configured to provide implementations for a same type of aggregation function that are defined differently by different data sources.
3 . The data query method of claim 1 , wherein processing the query description comprises:
determining whether the query field comprises a time data query field; and
processing the time data query field using a second component;
wherein the second component is configured to provide implementations for different formats or types of time data in different data sources.
4 . The data query method of claim 1 , wherein processing the query description comprises:
determining whether the query field comprises a time data query field;
processing the time data query field using a second component;
determining whether the query field comprises an aggregate query field; and
subsequent to processing the time data query field using the second component, processing the aggregate query field using a first component;
wherein the first component is configured to provide implementations for a same type of aggregation function that are defined differently by different data sources; and
the second component is configured to provide implementations for different formats or types of time data in different data sources.
5 . The data query method of claim 1 , wherein processing the filter type field comprises at least one of:
processing filter type field using a third component to generate a first filter sub-statement configured to filter character type fields;
processing filter type field using a fourth component to generate a second filter sub-statement configured to filter numeric type fields; or
processing filter type field using a fifth component to generate a third filter sub-statement configured to filter date data type fields;
wherein the filter sub-statement comprises at least one of the first filter sub-statement, the second filter sub-statement, or the third filter sub-statement.
6 . The data query method of claim 5 , further comprising:
obtaining filter condition field from the query description; or
obtaining the at least one data source type from the query description.
7 . The data query method of claim 1 , further comprising:
obtaining classification field from the query description; and
determining a third sub-statement.
8 . The data query method of claim 1 , further comprising:
obtaining order field from the query description; and
determining a fourth sub-statement.
9 . The data query method of claim 1 , wherein determining query statement comprises joining multiple sub-statements including the first sub-statement into the query statement.
10 . The data query method of claim 1 , further comprising:
loading the query statement onto a query engine; and
querying a data source, by the query engine, to obtain query result.
11 . The data query method of claim 10 , further comprising:
determining whether the query engine supports the data source;
determining whether querying the data source comprises a multi-table query;
upon a determination that querying the data source comprises the multi-table query, splitting the multi-table query into multiple single-table queries; and
assembling query results from the multiple single-table queries.
12 . The data query method of claim 10 , wherein the query engine is a Presto query engine.
13 . The data query method of claim 1 , further comprising:
loading an interface of a respective component of the one or more components onto the memory; and
obtaining respective implementations of the respective component from the interface of the respective component.
14 . The data query method of claim 1 , further comprising determining the query description;
wherein determining the query description comprises defining at least one of the query field, filter type field, filter condition field, and order field.
15 . The data query method of claim 1 , further comprising defining the one or more components;
wherein a respective component of the one or more components is defined to comprise a plurality of interfaces compatible with different data sources;
the respective component is configured to provide implementations for different formats or types of data in the different data sources.
16 . The data query method of claim 1 , wherein the one or more components comprise at least one user-defined component.
17 . The data query method of claim 1 , wherein the query statement is a SQL statement.
18 . A data query apparatus, comprising:
a memory;
one or more processors;
wherein the memory and the one or more processors are connected with each other; and
the memory stores computer-executable instructions for controlling the one or more processors to:
receive query description that specifies at least one data source type and at least one query parameter;
load an interface of at least one user-defined component onto the memory, obtaining a corresponding implementation of the at least one user-defined component, and grouping obtained implementations based on the at least one data source type;
process the query description using one or more components, by identifying the at least one data source type and executing data-source-specific operations associated with the at least one data source type to generate query statement that conforms to specified query parameters; and
dynamically adjust query statement based on the at least one data source type to enhance compatibility and processing efficiency;
wherein the one or more processors are configured to:
obtain query field from the query description;
process the query field using the one or more components;
determine a first sub-statement;
obtain filter type field from the query description that defines a specific subset of data types within the data source;
process the filter type field by loading an interface of a dedicated filtering component onto the memory, obtaining its implementation, and processing one or more dedicated filtering components configured to handle character, numeric, or date data types to generate a filter sub-statement that reflects the filtered results in a format compatible with the specified data source; and
determine a query statement by joining the first sub-statement with the filter sub-statement.
19 . A computer-program product comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon, the computer-readable instructions being executable by a processor to cause the processor to perform:
receiving query description that specifies at least one data source type and at least one query parameter;
loading an interface of at least one user-defined component onto a memory, obtaining a corresponding implementation of the at least one user-defined component, and grouping obtained implementations based on the at least one data source type;
processing the query description using one or more components, by identifying the at least one data source type and executing data-source-specific operations associated with the at least one data source type to generate query statement that conforms to specified query parameters; and
dynamically adjusting the query statement based on the at least one data source type to enhance compatibility and processing efficiency;
wherein processing the query description comprises:
obtaining query field from the query description;
processing the query field using the one or more components;
determining a first sub-statement;
obtaining filter type field from the query description that defines a specific subset of data types within the data source;
processing the filter type field by loading an interface of a dedicated filtering component onto the memory, obtaining its implementation, and processing one or more dedicated filtering components configured to handle character, numeric, or date data types to generate a filter sub-statement that reflects the filtered results in a format compatible with the specified data source; and
determining a query statement by joining the first sub-statement with the filter sub-statement.