Managing the processing of streamed data in a data streaming application using query information from a relational database

1. A computer-executed method, comprising:

monitoring queries against data in a computerized database to generate at least one parameter defining data of interest;

identifying selective in-flight data in a stream computing application which matches the at least one parameter defining the data of interest, the stream computing application producing output data for inclusion in the computerized database; and

responsive to said identifying selective in-flight data in the stream computing application which matches the at least one parameter defining the data of interest, modifying the manner in which the selective in-flight data is processed by the stream computing application.

2. The computer-executed method of claim 1 , wherein said modifying the manner in which the selective in-flight data is processed by the stream computing application comprises performing at least one additional processing action by the stream computing application on the selective in-flight data to produce at least one of enhanced data or enhanced metadata for output to the computerized database.

3. The computer-executed method of claim 2 , wherein performing at least one additional processing action to produce at least one of enhanced data or enhanced metadata comprises at least one of: (a) using at least one algorithm to provide greater accuracy of produced data or metadata; (b) using at least one algorithm to provide greater resolution of produced data or metadata; (c) using at least one algorithm to identify a pattern or feature of produced data or metadata; and (d) using at least one algorithm to identify and/or correct probable errors in the produced data or metadata.

4. The computer-executed method of claim 1 , wherein the at least one parameter defining data of interest comprises at least one logical query specifying the data of interest.

5. The computer-executed method of claim 1 , further comprising:

transmitting a meta-query command from the computerized database to the stream computing application, the meta-query command including a logical expression to be applied to at least one type of data tuple, the logical expression defining the data of interest, the meta-query command further specifying at least one action to be performed with respect each of a plurality of data tuples satisfying the conditions of the logical expression.

6. The computer-executed method of claim 1 , wherein said identifying selective in-flight data in the stream computing application which matches the at least one parameter defining the data of interest comprises:

receiving, in a management system of the stream computing application, a command containing the at least one parameter defining the data of interest;

sending, from the management system to a respective agent in each of at least one compute node of the stream computing application, a command containing the at least one parameter defining the data of interest;

identifying, in each of the at least one compute node, respective data matching the at least one parameter defining the data of interest.

7. The computer-executed method of claim 1 , further comprising:

collecting trace data from at least one execution instance of the stream computing application;

analyzing the collected trace data to produce profile data characterizing the stream computing application; and

using the profiling data to identify one or more locations in an operator graph of the stream computing application at which the selective in-flight data is identified.

8. The computer-executed method of claim 7 , wherein said profile data characterizing the stream computing application comprises a plurality of profile summary records, each profile summary record corresponding to a respective tuple and respective location of the respective tuple within the data streaming application.

9. The computer-executed method of claim 8 , wherein at least some of said profile summary records comprise a respective at least one stable field identifier identifying a respective field which does not change value from the respective location within the data streaming application and an output tuple.

10. A non-transitory computer-readable media having recorded thereon a computer program having a plurality of computer-executable instructions, wherein the computer program, when executed by at least one computer system, causes the at least one computer system to perform actions comprising:

monitoring queries against data in a computerized database to generate at least one parameter defining data of interest;

identifying selective in-flight data in a stream computing application which matches the at least one parameter defining the data of interest, the stream computing application producing output data for inclusion in the computerized database; and

responsive to said identifying selective in-flight data in the stream computing application which matches the at least one parameter defining the data of interest, modifying the manner in which the selective in-flight data is processed by the stream computing application.

11. The non-transitory computer-readable media of claim 10 , wherein modifying the manner in which the selective in-flight data is processed by the stream computing application comprises performing at least on additional processing action by the stream computing application on the selective in-flight data to produce at least one of enhanced data or enhanced metadata for output to the computerized database.

12. The non-transitory computer-readable media of claim 11 , wherein performing at least one additional processing action to produce at least one of enhanced data or enhanced metadata comprises at least one of: (a) using at least one algorithm to provide greater accuracy of produced data or metadata; (b) using at least one algorithm to provide greater resolution of produced data or metadata; (c) using at least one algorithm to identify a pattern or feature of produced data or metadata; and (d) using at least one algorithm to identify and/or correct probable errors in the produced data or metadata.

13. The non-transitory computer-readable media of claim 10 , wherein the at least one parameter defining data of interest comprises at least one logical query specifying the data of interest.

14. The non-transitory computer-readable media of claim 10 , further comprising:

transmitting a meta-query command from the computerized database to the stream computing application, the meta-query command including a logical expression to be applied to at least one type of data tuple, the logical expression defining the data of interest, the meta-query command further specifying at least one action to be performed with respect each of a plurality of data tuples satisfying the conditions of the logical expression.

15. The non-transitory computer-readable media of claim 10 , wherein said identifying selective in-flight data in the stream computing application which matches the at least one parameter defining the data of interest comprises:

receiving, in a management system of the stream computing application, a command containing the at least one parameter defining the data of interest;

sending, from the management system to a respective agent in each of at least one compute node of the stream computing application, a command containing the at least one parameter defining the data of interest;

identifying, in each of the at least one compute node, respective data matching the at least one parameter defining the data of interest.

16. The non-transitory computer-readable media of claim 10 , wherein the computer program further causes the computer system to perform actions comprising:

collecting trace data from at least one execution instance of the stream computing application;

analyzing the collected trace data to produce profile data characterizing the stream computing application; and

using the profiling data to identify one or more locations in an operator graph of the stream computing application at which the selective in-flight data is identified.

17. The non-transitory computer-readable media of claim 16 , wherein said profile data characterizing the stream computing application comprises a plurality of profile summary records, each profile summary record corresponding to a respective tuple and respective location of the respective tuple within the data streaming application.

18. The non-transitory computer-readable media of claim 17 , wherein at least some of said profile summary records comprise a respective at least one stable field identifier identifying a respective field which does not change value from the respective location within the data streaming application and an output tuple.

19. A data processing apparatus supporting a database, the database receiving input from a stream computing application, the data processing apparatus comprising:

at least one physical processor;

at least one physical memory;

at least one database table containing a plurality of tuples storable in the memory;

a database manager embodied as instructions executable on the at least one processor and storable in the at least one memory, the database manager including a query function for executing logical queries against data in the at least one database table, and a query monitor which monitors the logical queries to generate at least one parameter defining data of interest;

wherein the data base manager, responsive to the query monitor generating at least one parameter defining data of interest, transmits a command to the stream computing application to identify selective in-flight data in the stream computing application which matches the at least one parameter defining the data of interest and to modify the manner in which the selective in-flight data is processed by the stream computing application; and

wherein the stream computing application, responsive to receiving the command to to identify selective in-flight data in the stream computing application which matches the at least one parameter defining the data of interest and to modify the manner in which the selective in-flight data is processed by the stream computing application, identifies selective in-flight data in the stream computing application which matches the at least one parameter defining the data of interest, and modifies the manner in which the selective in-flight data is processed by the stream computing application.

20. The data processing apparatus of claim 19 , wherein modifying the manner in which the selective in-flight data is processed by the stream computing application comprises performing at least on additional processing action by the stream computing application on the selective in-flight data to produce at least one of enhanced data or enhanced metadata for output to the computerized database.