Streams analysis tool and method
A streams analysis tool allows a user to define one or more buckets according to a specified tuple collection criteria for each bucket. The specified tuple collection criteria for each bucket defines some way to distinguish one data tuple from another. The specified tuple collection criteria for each bucket is therefore used to distinguish data tuples that satisfy the specified tuple collection criteria from data tuples that do not satisfy the specified tuple collection criteria. When a data tuple satisfies the specified tuple collection criteria for a bucket, the data tuple is stored in the bucket. In addition, data tuples preceding or succeeding the data tuple may also be stored in the bucket, as determined by the specified tuple collection criteria. The data tuples in each bucket are analyzed, and based on the analysis a streams manager can change how future data tuples are processed by the streaming application.
1. An apparatus comprising:
at least one processor;
a memory coupled to the at least one processor;
a streams manager residing in the memory and executed by the at least one processor, the streams manager executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples; and
a streams analysis tool residing in the memory and executed by the at least one processor that comprises a first bucket defined by a user that specifies first tuple collection criteria for distinguishing some of the plurality of data tuples in the streaming application from other of the plurality of data tuples in the streaming application, the streams analysis tool analyzing the plurality of data tuples as the streaming application is executed by the streams manager and storing each of the plurality of data tuples that satisfies the first tuple collection criteria in the first bucket, wherein the streams analysis tool analyzes data tuples in the first bucket and feeds back information from analyzing the data tuples in the first bucket to the streams manager to change how the streaming application processes future data tuples, and wherein, in response to the information fed back from the streams analysis tool, the streams manager causes filtering of at least one data tuple in the flow graph.
2. The apparatus of claim 1 wherein the first tuple collection criteria specifies at least one data value or range.
3. The apparatus of claim 1 wherein the first tuple collection criteria specifies at least one metadata value or range.
4. The apparatus of claim 1 wherein the first tuple collection criteria specifies a time range.
5. The apparatus of claim 1 wherein the first tuple collection criteria specifies at least one event.
6. The apparatus of claim 1 wherein the first tuple collection criteria specifies a first number of tuples preceding a matching data tuple and a second number of tuples succeeding the matching data tuple.
7. The apparatus of claim 1 wherein the streams analysis tool comprises a second bucket that specifies second tuple collection criteria, and wherein the first tuple collection criteria and the second tuple collection criteria are user-defined.
8. An apparatus comprising:
at least one processor;
a memory coupled to the at least one processor;
a streams manager residing in the memory and executed by the at least one processor, the streams manager executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples; and
a streams analysis tool residing in the memory and executed by the at least one processor that comprises a first bucket defined by a user that specifies first tuple collection criteria for distinguishing some of the plurality of data tuples in the streaming application from other of the plurality of data tuples in the streaming application, the streams analysis tool analyzing the plurality of data tuples as the streaming application is executed by the streams manager and storing each of the plurality of data tuples that satisfies the first tuple collection criteria in the first bucket, wherein the streams analysis tool analyzes data tuples in the first bucket and feeds back information from analyzing the data tuples in the first bucket to the streams manager to change how the streaming application processes future data tuples, and wherein, in response to the information fed back from the streams analysis tool, the streams manager prioritizes processing of at least one data tuple in the flow graph.
9. An apparatus comprising:
at least one processor;
a memory coupled to the at least one processor;
a streams manager residing in the memory and executed by the at least one processor, the streams manager executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples; and
a streams analysis tool residing in the memory and executed by the at least one processor that comprises a first bucket defined by a user that specifies first tuple collection criteria for distinguishing some of the plurality of data tuples in the streaming application from other of the plurality of data tuples in the streaming application, wherein the first tuple collection criteria comprises:
at least one data value or range;
at least one time range; and
a first number of tuples preceding a matching data tuple and a second number of tuples succeeding the matching data tuple;
wherein the streams analysis tool further comprises a second bucket defined by the user that specifies second tuple collection criteria, wherein the second tuple collection criteria comprises:
at least one metadata value or range; and
at least one event;
wherein the streams analysis tool analyzes the plurality of data tuples as the streaming application is executed by the streams manager, stores each data tuple of the plurality of data tuples that satisfies the first tuple collection criteria in the first bucket, stores each data tuple of the plurality of data tuples that satisfies the second tuple collection criteria in the second bucket, analyzes the plurality of data tuples in the first bucket, analyzes the plurality of data tuples in the second bucket, and feeds back information from analyzing the plurality of data tuples in the first bucket and the second bucket to the streams manager to change how the streaming application processes future data tuples, wherein, in response to the information fed back from the streams analysis tool, the streams manager performs at least one of:
filtering at least one data tuple in the flow graph; and
prioritizing processing of at least one data tuple in the flow graph.