Addition and processing of continuous SQL queries in a streaming relational database management system
Systems, methods, and media are disclosed herein that can be embodied in a traditional Relational Database Management System (RDBMS) in order to transform it into a Streaming Relational Database Management System (SRDBMS). An SRDBMS may provide functionality such as to manage and populate streams, tables, and archived stream histories and support the evaluation of continuous queries on streams and tables. Both continuous and snapshot queries support the full spectrum of the industry standard, widely used, Structured Query Language. The present technology can support a high number of concurrent continuous queries using a scalable and efficient shared query evaluation scheme, support on-the-fly addition of continuous queries into a mechanism that implements the shared evaluation scheme, reuse RDBMS modules such as relational operators and expression evaluators, and visualize results of continuous queries in real time.
1. A method for managing a data stream, comprising:
receiving a continuous query;
generating an iterator-model plan based on the continuous query;
adding the continuous query to a shared continuous-query plan by:
analyzing the iterator-model plan and generating a plurality of iterator-model subplans by walking the iterator-model plan bottom-up to generate transforms, plan items and qual items based on the iterator-model plan; and
for at least one particular iterator-model subplan of the plurality of iterator-model subplans:
assigning a continuous-query operator to the particular iterator-model subplan; and
adding an adapter as a new leaf node to the particular iterator-model subplan, wherein the adapter is an iterator-model operator configured to fetch one or more processed continuous-query tuples from the continuous-query operator and provide the one or more fetched continuous-query tuples to the particular iterator-model subplan of the adapter;
applying the transforms to convert local range variables of the continuous query to global range variables; and
based on the plan items and the qual items, modifying the shared continuous-query plan, wherein the plan items refer to one or more new or existing continuous-query operator of the shared continuous-query plan, and wherein the qual items comprise information specific to at least one of qualifiers and predicates;
wherein the method is performed by one or more computing devices.
2. The method of claim 1 , further comprising:
providing the one or more processed continuous-query tuples to the particular iterator-model subplan.
3. The method of claim 1 , wherein the continuous-query operator operates on a stream.
4. The method of claim 1 , wherein the shared continuous-query plan comprises a routing table encoding a plurality of routes for tuples in a shared dataflow, wherein adding the continuous query to the shared continuous-query plan further comprises modifying the routing table.
5. A non-transitory computer-readable medium storing instructions which, when processed by one or more processors, cause:
receiving a continuous query;
generating an iterator-model plan based on the continuous query;
adding the continuous query to a shared continuous-query plan by:
analyzing the iterator-model plan and generating a plurality of iterator-model subplans by walking the iterator-model plan bottom-up to generate transforms, plan items and qual items based on the iterator-model plan; and
for at least one particular iterator-model subplan of the plurality of iterator-model subplans:
assigning a continuous-query operator to the particular iterator-model subplan; and
adding an adapter as a new leaf node to the particular iterator-model subplan, wherein the adapter is an iterator-model operator configured to fetch one or more processed continuous-query tuples from the continuous-query operator and provide the one or more fetched continuous-query tuples to the particular iterator-model subplan of the adapter;
applying the transforms to convert local range variables of the continuous query to global range variables; and
based on the plan items and the qual items, modifying the shared continuous-query plan, wherein the plan items refer to one or more new or existing continuous-query operator of the shared continuous-query plan, and wherein the qual items comprise information specific to at least one of qualifiers and predicates.
6. The non-transitory computer-readable medium as recited in claim 5 , further comprising providing the one or more processed continuous-query tuples to the particular iterator model subplan.
7. The non-transitory computer-readable medium as recited in claim 5 , wherein the continuous-query operator operates on a stream.
8. The non-transitory computer-readable medium as recited in claim 5 , wherein the shared continuous-query plan comprises a routing table encoding a plurality of routes for tuples in a shared dataflow, wherein adding the continuous query to the shared continuous-query plan further comprises modifying the routing table.