DATA STREAM PROCESSING BASED ON A BOUNDARY PARAMETER
In one implementation, a system for processing a data stream can comprise a station engine, an execution engine, and a synchronize engine. A station engine can provide a stream operator to receive application logic, punctuate the data stream, and determine a number of input channels for parallel processing. The execution engine can perform a behavior of the application logic during a process operation. The synchronize engine can hold data of the data stream associated with a window until each input channel has reached a data boundary based on a boundary parameter.
1 . A system for processing a data stream comprising:
a station engine to provide a stream operator to:
receive application logic for sliding window processing;
punctuate the data stream based on a boundary parameter; and
determine a number of input channels for parallel processing;
an execution engine to perform a behavior of the application logic during a process operation; and
a synchronize engine to hold data of the data stream associated with a window until each input channel has reached a data boundary based on the boundary parameter.
2 . The system of claim 1 , wherein the execution engine is to:
perform the behavior of the application logic based on a plurality of boundary parameters, wherein the plurality of boundary parameters comprises:
a granule size to be a range of tuples;
a slide size to be a first range of granules; and
a window size to be a second range of granules.
3 . The system of claim 2 , wherein, based on the data boundary, the behavior is to summarize one of a window, a slide, and a granule in accordance with the application logic.
4 . The system of claim 1 , comprising:
a spout engine to generate tuples with a granule field;
the synchronize engine to maintain a granule table to contain a current granule number of each input channel.
5 . The system of claim 4 , comprising:
a combine engine to combine the output of a set of summaries based on the conclusion of the window, the conclusion based on the granule table.
6 . A machine readable storage medium comprising a set of instructions executable by a processor resource to:
execute a template behavior to initialize parallel processing of a data stream;
execute a dynamic behavior based on a boundary parameter and application logic for sliding window processing,
hold a tuple of the data stream when a granule number of the current input is larger than a resolved granule number; and
a process the held tuple of a first window based on the application logic when a second window boundary is achieved.
7 . The medium of claim 6 , wherein the set of instructions is to:
receive the application logic to specify processing details of a template logic.
8 . The medium of claim 6 , wherein the set of instructions is to:
partially process the first window based on a punctuation of a set of held tuples, the set of held tuples being less than a window size.
9 . The medium of claim 6 , wherein the set of instructions is to:
process the first window when a first window boundary is achieved and a slide boundary is achieved.
10 . The medium of claim 6 , wherein the set of instructions is to:
resolve a least granule number from an input channel; and
hold the tuple when a slide operation does not advance.
11 . A method for processing a data stream comprising:
receiving boundary parameters including a granule size to be a range of tuples, a slide size to be a number of granules, and a window size to be a number of granules;
invoking application logic to process the data stream based on a sliding window technique, the application logic to be plugged into template logic;
receiving input from one of a plurality of channels, the data stream to be a processed by the plurality of channels based on the application logic;
holding a tuple when a current input is larger than a resolved input; and
processing a tuple when a punctuation boundary is achieved.
12 . The method of claim 11 , comprising:
determining a level of processing based on a set of held tuples, the boundary parameters, and the application logic.
13 . The method of claim 12 , wherein the level of processing is a partial window processing when the set of held tuples is less than the window size.
14 . The method of claim 13 , wherein the level of processing is a slide summarization when the set of held tuples achieves a slide boundary.
15 . The method of claim 13 , wherein the level of processing is a granule summarization when the set of held tuples achieves a granule boundary.