Data source tokens
A data intake and query system processes and stores events, which are associated with token identifiers for tokens corresponding to data sources for the messages that the events are generated from. Thus, the data intake and query system can receive a request to provide analyses and visualizations regarding stored events associated with a particular component associated with a plurality of events, such as a data source for the messages from which the plurality of events are generated from. These requests and the resulting visualizations can be customized based on selected tokens and selected components.
1. A computer-implemented method, comprising:
identifying a plurality of events associated with a token identifier, the plurality of events received via a plurality of messages and stored by a data intake and query system, wherein the token identifier indicates a data source of the plurality of messages;
identifying one or more components of the data intake and query system that process the plurality of events received via the plurality of messages; and
generating visualization data for a graphical visualization, the visualization data indicating one or more metrics of the one or more components of the data intake and query system that process the plurality of events received via the plurality of messages.
2. The computer-implemented method of claim 1 , wherein the one or more components of the data intake and query system comprise a plurality of event collectors, wherein the plurality of event collectors are configured to receive the plurality of events received via the plurality of messages and manage storage of the plurality of events received via the plurality of messages.
3. The computer-implemented method of claim 1 , wherein the one or more components of the data intake and query system comprise a plurality of indexers, wherein the plurality of indexers are configured to store the plurality of events received via the plurality of messages.
4. The computer-implemented method of claim 1 , wherein the one or more components of the data intake and query system comprise a plurality of forwarders, wherein the plurality of forwarders are configured to receive the plurality of events received via the plurality of messages and forward the plurality of events received via the plurality of messages for storage.
5. The computer-implemented method of claim 1 , wherein the one or more metrics relate to an operation of the one or more components of the data intake and query system.
6. The computer-implemented method of claim 1 , wherein the one or more metrics relate to an operation of the one or more components of the data intake and query system, and wherein the one or more metrics comprise CPU-related performance metrics, disk-related performance metrics, memory-related performance metrics, network-related performance metrics, energy-usage statistics, data-traffic-related performance metrics, overall system availability performance metrics, cluster-related performance metrics, or virtual machine performance statistics.
7. The computer-implemented method of claim 1 , wherein the plurality of events received via the plurality of messages are generated based on the plurality of messages.
8. The computer-implemented method of claim 1 , wherein the token identifier corresponds to a token, wherein the token comprises metadata for the data source.
9. The computer-implemented method of claim 1 , wherein the token identifier corresponds to a token, wherein the token comprises timestamps for data in the plurality of messages.
10. The computer-implemented method of claim 1 , wherein the token identifier is configured to identify a token associated with both the data source and the one or more components of the data intake and query system.
11. The computer-implemented method of claim 1 , wherein the token identifier comprises a user-provided token name for a token.
12. The computer-implemented method of claim 1 , wherein the token identifier corresponds to a token that is included in each message of the plurality of messages.
13. The computer-implemented method of claim 1 , wherein the token identifier corresponds to a token that provides metadata for the plurality of events received via the plurality of messages.
14. The computer-implemented method of claim 1 , wherein the visualization data relates to a percentage of the plurality of events received via the plurality of messages that were successfully acknowledged for the one or more components of the data intake and query system.
15. The computer-implemented method of claim 1 , wherein the visualization data relates to a percentage of the plurality of events received via the plurality of messages that were successfully acknowledged for a token corresponding to the token identifier.
16. The computer-implemented method of claim 1 , wherein the graphical visualization comprises a bar chart, scatter plot, area chart, line chart, pie chart, radial gauge, marker gauge, or filler gauge.
17. The computer-implemented method of claim 1 , wherein the visualization data is based on per-token metrics and system-wide metrics.
18. The computer-implemented method of claim 1 , wherein the data source provides the plurality of messages to the one or more components of the data intake and query system.
19. The computer-implemented method of claim 1 , wherein the data intake and query system receives the plurality of messages from the data source and routes the plurality of messages to the one or more components of the data intake and query system.
20. The computer-implemented method of claim 1 , wherein the data source provides the plurality of messages to the one or more components of the data intake and query system, and wherein the visualization data is related to the data source.
21. The computer-implemented method of claim 1 , wherein each of the one or more components of the data intake and query system store the token identifier.
22. The computer-implemented method of claim 1 , wherein each of the plurality of messages comprises a token corresponding to the token identifier, raw machine data, and event metadata.
23. The computer-implemented method of claim 1 , further comprising:
calculating values associated with the visualization data;
comparing the values to a threshold; and
invoking a process based on comparing the values to the threshold.
24. A computing system, comprising:
one or more processing devices configured to:
identify a plurality of events associated with a token identifier, the plurality of events received via a plurality of messages and stored by a data intake and query system, wherein the token identifier indicates a data source of the plurality of messages;
identify one or more components of the data intake and query system that process the plurality of events received via the plurality of messages; and
generate visualization data for a graphical visualization, the visualization data indicating one or more metrics of the one or more components of the data intake and query system that process the plurality of events received via the plurality of messages.
25. The computing system of claim 24 , wherein the plurality of events received via the plurality of messages are generated based on the plurality of messages.
26. The computing system of claim 24 , wherein the token identifier corresponds to a token, wherein the token comprises metadata for the data source.
27. The computing system of claim 24 , wherein the token identifier is configured to identify a token associated with both the data source and the one or more components of the data intake and query system.
28. Non-transitory computer readable storage media comprising computer-executable instructions that, when executed by a computing system, cause the computing system to:
identify a plurality of events associated with a token identifier, the plurality of events received via a plurality of messages and stored by a data intake and query system, wherein the token identifier indicates a data source of the plurality of messages;
identify one or more components of the data intake and query system that process the plurality of events received via the plurality of messages; and
generate visualization data for a graphical visualization, the visualization data indicating one or more metrics of the one or more components of the data intake and query system that process the plurality of events received via the plurality of messages.
29. The non-transitory computer readable storage media of claim 28 , wherein the plurality of events received via the plurality of messages are generated based on the plurality of messages.
30. The non-transitory computer readable storage media of claim 28 , wherein the token identifier corresponds to a token, wherein the token comprises metadata for the data source.