Streams analysis tool and method
A streams analysis tool allows a user to define one or more buckets according to a specified tuple collection criteria for each bucket. The specified tuple collection criteria for each bucket defines some way to distinguish one data tuple from another. The specified tuple collection criteria for each bucket is therefore used to distinguish data tuples that satisfy the specified tuple collection criteria from data tuples that do not satisfy the specified tuple collection criteria. When a data tuple satisfies the specified tuple collection criteria for a bucket, the data tuple is stored in the bucket. In addition, data tuples preceding or succeeding the data tuple may also be stored in the bucket, as determined by the specified tuple collection criteria. The data tuples in each bucket are analyzed, and based on the analysis a streams manager can change how future data tuples are processed by the streaming application.
1. A computer-implemented method executed by at least one processor for running streaming applications, the computer-implemented method comprising:
executing a streams manager that executes a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples;
a user defining a first bucket that specifies first tuple collection criteria for distinguishing some of the plurality of data tuples in the streaming application from other of the plurality of data tuples in the streaming application;
analyzing the plurality of data tuples as the streaming application is executed by the streams manager;
storing each of the plurality of data tuples that satisfies the first tuple collection criteria in the first bucket;
analyzing data tuples in the first bucket; and
feeding back information from analyzing the data tuples in the first bucket to the streams manager to change how the streaming application processes future data tuples, wherein, in response to the information fed back from the streams analysis tool, the streams manager causes filtering of at least one data tuple in the flow graph.
2. The computer-implemented method of claim 1 wherein the first tuple collection criteria specifies at least one data value or range.
3. The computer-implemented method of claim 1 wherein the first tuple collection criteria specifies at least one metadata value or range.
4. The computer-implemented method of claim 1 wherein the first tuple collection criteria specifies a time range.
5. The computer-implemented method of claim 1 wherein the first tuple collection criteria specifies at least one event.
6. The computer-implemented method of claim 1 wherein the first tuple collection criteria specifies a first number of tuples preceding a matching data tuple and a second number of tuples succeeding the matching data tuple.
7. The computer-implemented method of claim 1 further comprising:
defining a second bucket that specifies second tuple collection criteria[N] z wherein the first tuple collection criteria and the second tuple collection criteria are user-defined.
8. A method for analyzing a streaming application, the method comprising:
executing a streams manager that executes a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples;
a user defining a first bucket that specifies first tuple collection criteria for distinguishing some of the plurality of data tuples in the streaming application from other of the plurality of data tuples in the streaming application, wherein the first tuple collection criteria comprises:
at least one data value or range;
at least one time range; and
a first number of tuples preceding a matching data tuple and a second number of tuples succeeding the matching data tuple;
the user defining a second bucket that specifies second tuple collection criteria, wherein the second tuple collection criteria comprises:
at least one metadata value or range; and
at least one event;
analyzing the plurality of data tuples as the streaming application is executed by the streams manager;
storing each data tuple of the plurality of data tuples that satisfies the first tuple collection criteria in the first bucket;
storing each data tuple of the plurality of data tuples that satisfies the second tuple collection criteria in the second bucket;
analyzing the plurality of data tuples in the first bucket;
analyzing the plurality of data tuples in the second bucket; and
feeding back information from analyzing the plurality of data tuples in the first bucket and the second bucket to the streams manager to change how the streaming application processes future data tuples, wherein, in response to the information fed back from the streams analysis tool, the streams manager performs at least one of:
filtering at least one data tuple in the flow graph; and
prioritizing processing of at least one data tuple in the flow graph.
9. A computer-implemented method executed by at least one processor for running streaming applications, the computer-implemented method comprising:
executing a streams manager that executes a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples;
a user defining a first bucket that specifies first tuple collection criteria for distinguishing some of the plurality of data tuples in the streaming application from other of the plurality of data tuples in the streaming application;
analyzing the plurality of data tuples as the streaming application is executed by the streams manager;
storing each of the plurality of data tuples that satisfies the first tuple collection criteria in the first bucket;
analyzing data tuples in the first bucket; and
feeding back information from analyzing the data tuples in the first bucket to the streams manager to change how the streaming application processes future data tuples, wherein, in response to the information fed back from the streams analysis tool, the streams manager prioritizes processing of at least one data tuple in the flow graph.