Apparatus and method for processing streaming data and forming visualizations thereof
A machine has a bus and a network interface circuit to receive different data streams from a network. The network interface circuit is connected to the network and the bus. A processor is connected to the bus. A memory is connected to the bus. The memory stores instructions executed by the processor to continuously increment aggregate functions associated with data parameters within the different data streams. Visualizations of the different data streams are periodically updated on different client devices connected to the network.
1. A machine, comprising;
a bus;
a network interface circuit to receive different data streams from a network, the network interface circuit being connected to the network and the bus;
a processor connected to the bus; and
a memory connected to the bus, the memory storing instructions executed by the processor to:
continuously increment aggregate functions associated with data parameters within the different data streams,
define a visualization priority threshold tuned depending on a type of visualization of the different data streams,
periodically update visualizations of the different data streams on different client devices connected to the network, wherein an update occurs when the visualization priority threshold is reached, and wherein the different data streams are retrieved from different data stream servers connected to the network, and wherein the visualization priority threshold comprises one of: an amount of data of the data streams to collect, or an amount of time of data collection, and
handle configuration parameters with the different data stream servers, and further wherein the visualization priority threshold is tuned to provide near continuous updates to a heat map type visualization, and non-continuous updates to a bar or line chart type visualization.
2. The machine of claim 1 further comprising instructions executed by the processor to track currently executing queries and currently active data consumers of the different data streams.
3. The machine of claim 1 further comprising instructions executed by the processor to manage requests for data from the different data stores from multiple data stores.
4. The machine of claim 1 further comprising instructions executed by the processor to manage requests for data from the different data stores from multiple consumers.
5. The machine of claim 1 wherein the aggregate functions include a sum, a count, a minimum and a maximum.
6. The machine of claim 1 further comprising instructions executed by the processor to combine aggregate functions to compute statistical measures.
7. The machine of claim 6 further comprising instructions executed by the processor to incorporate the aggregate functions and statistical measures in visualizations of the data streams.
8. A method comprising:
continuously incrementing aggregate functions associated with data parameters within the different data streams;
defining a visualization priority threshold tuned depending on a type of visualization of the different data streams, and
periodically updating visualizations of the different data streams on different client devices connected to the network, wherein an update occurs when the visualization priority threshold is reached, and wherein the different data streams are retrieved from different data stream servers connected to the network, and wherein the visualization priority threshold comprises one of: an amount of data of the data streams to collect, or an amount of time of data collection, and
handling configuration parameters with the different data stream servers, and further wherein the visualization priority threshold is tuned to provide near continuous updates to a heat map type visualization, and non-continuous updates to a bar or line chart type visualization.
9. The method of claim 8 further comprising tracking currently executing queries and currently active data consumers of the different data streams.
10. The method of claim 8 further comprising managing requests for data from the different data stores from multiple data stores.
11. The method of claim 8 further comprising managing requests for data from the different data stores from multiple consumers.
12. The method of claim 8 wherein the aggregate functions include a sum, a count, a minimum and a maximum.
13. The method of claim 8 further comprising combining aggregate functions to compute statistical measures.
14. The method of claim 13 further comprising incorporating the aggregate functions and statistical measures in visualizations of the data streams.