Processing a data stream
View Patent ↗Methods, database management systems (“DBMS”) and computer-readable media are provided for processing unbounded stream data using a traditional DBMS. Execution of a query that includes a data stream as a data source may be initiated. Tuples may be processed in accordance with the query as the tuples are received through the data stream until an indication is received that execution of the query should cease.
1. A method of processing a data stream, comprising:
initiating execution of a query that includes the data stream as a data source, wherein the query specifies an invocation pattern for the data stream; and
processing tuples in accordance with the query as the tuples are received through the data stream until an indication is received that execution of the query should cease, wherein the data stream is included in the query as a relation-valued function that returns one or more tuples in a return mode based on the invocation pattern, wherein the return mode comprises one of a tuple mode returning one tuple and a set mode returning multiple tuples.
2. The method of claim 1 , wherein the data stream is included in the query as a stream source function that returns a tuple in a format suitable for an operator defined in the query.
3. The method of claim 1 , wherein the relation-valued function is event-driven.
4. The method of claim 1 , wherein the relation-valued function returns a tuple upon an occurrence of one of a predetermined number of events or passage of a predetermined amount of time.
5. The method of claim 1 , further comprising:
initiating, at a query executor configured to execute the query, a handle to memory allocated to a data structure for use by the relation-valued function; and
passing the handle from the query executor to the relation-valued function.
6. The method of claim 5 , wherein the handle references memory that is used to store a state of the relation-valued function.
7. The method of claim 5 , further comprising:
storing a tuple returned from the relation-valued function in the data structure; and
passing the handle from the relation-valued function to the query executor.
8. The method of claim 5 , wherein the indication that execution of the query should cease includes an end-of-data operator.
9. The method of claim 8 , further comprising:
passing the end-of-data operator from the relation-valued function to the query executor; and
terminating execution of the query at the query executor.
10. The method of claim 8 , wherein the end-of-data operator is defined in the data structure.
11. A database management system configured to process data streams, comprising:
a query executor executing on one or more hardware processors and being configured to:
receive a query that includes a data stream as a data source, wherein the data stream is included in the query as a relation-valued function, and wherein the query specifies an invocation pattern for the data stream; and
process tuples based on the invocation pattern, as they are returned from a function manager executing the relation-valued function, in accordance with the query;
a function manager executing on one or more processors and being configured to:
execute the relation-valued function included in the query to return one or more tuples to the query executor in a return mode based on the invocation pattern, wherein the return mode comprises one of a tuple mode returning one tuple and a set mode returning multiple tuples.
12. The database management system of claim 11 , wherein the query executor is further configured to initiate a data structure for use by the function manager in executing the relation-valued function, and the function manager is configured to receive a handle to the data structure as input for the relation-valued function.
13. The database management system of claim 11 , wherein the query executor is further configured to process tuples as they are returned by the function manager from the relation-valued function until an indication is received that the query should cease.
14. The database management system of claim 13 , wherein the relation-valued function receives input data from the data stream.
15. The database management system of claim 11 , wherein the relation-valued function is an event-driven function.
16. The database management system of claim 15 , wherein the relation-valued function returns a tuple upon an occurrence of one of a predetermined number of events or passage of a predetermined amount of time.
17. A non-transitory computer-readable storage medium having computer-executable instructions for executing traditional database queries using a data stream as a data source, the instructions causing a computer to perform steps comprising:
initiating execution of a traditional database query that includes, as a data source, a relation-valued function which receives a data stream as input, wherein the traditional database query specifies an invocation pattern for the data stream; and
processing one or more tuples as they are returned from the relation-valued function in accordance with an operator defined in the traditional database query, and in a return mode based on the invocation pattern, until the relation-valued function returns an indication that the traditional database query should cease, wherein the return mode comprises one of a tuple mode returning one tuple and a set mode returning multiple tuples.
18. The computer-readable storage medium of claim 17 , further including computer-executable instructions for:
initiating, at a query executor configured to process the tuples in accordance with the operator defined in the traditional database query, a handle to a data structure for use by the relation-valued function; and
passing the handle from the query executor to a function manager that executes the relation-valued function.
19. The computer-readable storage medium of claim 17 , wherein the relation-valued function returns a tuple upon an occurrence of one of a predetermined number of events or passage of a predetermined amount of time.