Data enrichment and augmentation
Data enrichment and augmentation is disclosed. Machine data comprising at least one of a log message and a metrics data point is received. The received machine data comprises an identifier of an instance of a virtual machine. Based at least in part on the identifier of the instance of the virtual machine, a query for tags associated with the instance of the virtual machine is performed. At least one key-value pair is generated based at least in part on tags received in response to the query performed based at least in part on the identifier of the instance of the virtual machine. The received machine data is augmented with the at least one key-value pair generated based at least in part on the tags received in response to the query based at least in part on the identifier of the instance of the virtual machine.
1. A computer-implemented method comprising:
providing a user interface (UI) for presenting information on log messages and time series based on information received from one or more monitored devices;
in response to a user selection of a first time series, presenting the first time series in the UI;
detecting a request for information about log messages associated with the first time series; and
in response to detecting the request for information:
retrieving, from a time series data store, metadata associated with the first time series, the metadata comprising log message identifiers;
selecting, from a log data store, log messages based on the log message identifiers in the retrieved metadata, the log messages comprising time stamps;
assigning the log messages, based on the time stamps, to buckets corresponding to respective predetermined time intervals;
determining, for each bucket, a count of log messages and an associated intensity classification based on the count of log messages; and
presenting the selected log messages in the UI, the UI including an option to overlay information from the first time series and an intensity plot comprising intensity classifications associated with the buckets, wherein metrics data points of the first time series have an identifying metadata and are derived from the loci messages for a metric name in the loci message, wherein metrics data points with a different identifying metadata that are derived from additional loci messages for the metric name are stored in a second time series.
2. The method as recited in claim 1 , further comprising:
detecting hovering in the UI over a graphical presentation of the first time series; and
showing log messages associated with the first time series in response to detecting the hovering.
3. The method as recited in claim 1 , wherein the UI further includes an option to select multiple time series.
4. The method as recited in claim 1 , further comprising:
providing an option in the UI for entering additional metadata for searching the log messages associated with the first time series.
5. The method as recited in claim 1 , further comprising:
providing an option in the UI for entering additional log search terms for searching the log messages associated with the first time series.
6. The method as recited in claim 1 , wherein the UI further includes presenting a log message when a mouse pointer hovers over the first time series.
7. The method as recited in claim 1 , wherein the UI further includes presenting a log message count associated with the first time series.
8. The method as recited in claim 1 , wherein the log messages further comprise a metric name, a metric value corresponding to the metric name, a field name, and a field value corresponding to the field name.
9. A system comprising:
a memory comprising instructions; and
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:
providing a user interface (UI) for presenting information on log messages and time series based on information received from one or more monitored devices;
in response to a user selection of a first time series, presenting the first time series in the UI;
detecting a request for information about log messages associated with the first time series; and
in response to detecting the request for information:
retrieving, from a time series data store, metadata associated with the first time series, the metadata comprising log message identifiers;
selecting, from a log data store, log messages based on the log message identifiers in the retrieved metadata, the log messages comprising time stamps;
assigning the log messages, based on the time stamps, to buckets corresponding to respective predetermined time intervals;
determining, for each bucket, a count of log messages and an associated intensity classification based on the count of log messages; and
presenting the selected log messages in the UI, the UI including an option to overlay information from the first time series and an intensity plot comprising intensity classifications associated with the buckets,
wherein metrics data points of the first time series have an identifying metadata and are derived from the loci messages for a metric name in the loci message, wherein metrics data points with a different identifying metadata that are derived from additional log messages for the metric name are stored in a second time series.
10. The system as recited in claim 9 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
detecting hovering in the UI over a graphical presentation of the first time series; and
showing log messages associated with the first time series in response to detecting the hovering.
11. The system as recited in claim 9 , wherein the UI further includes an option to select multiple time series.
12. The system as recited in claim 9 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
providing an option in the UI for entering additional metadata for searching the log messages associated with the first time series.
13. The system as recited in claim 9 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
providing an option in the UI for entering additional log search terms for searching the log messages associated with the first time series.
14. A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
providing a user interface (UI) for presenting information on log messages and time series based on information received from one or more monitored devices;
in response to a user selection of a first time series, presenting the first time series in the UI;
detecting a request for information about log messages associated with the first time series; and
in response to detecting the request for information:
retrieving, from a time series data store, metadata associated with the first time series, the metadata comprising log message identifiers;
selecting, from a log data store, log messages based on the log message identifiers in the retrieved metadata, the log messages comprising time stamps;
assigning the log messages, based on the time stamps, to buckets corresponding to respective predetermined time intervals;
determining, for each bucket, a count of log messages and an associated intensity classification based on the count of log messages; and
presenting the selected log messages in the UI, the UI including an option to overlay information from the first time series and an intensity plot comprising intensity classifications associated with the buckets, wherein metrics data points of the first time series have an identifying metadata and are derived from the loci messages for a metric name in the loci message, wherein metrics data points with a different identifying metadata that are derived from additional loci messages for the metric name are stored in a second time series.
15. The non-transitory machine-readable storage medium as recited in claim 14 , wherein the machine further performs operations comprising:
detecting hovering in the UI over a presentation of the first time series; and
showing log messages associated with the first time series in response to detecting the hovering.
16. The non-transitory machine-readable storage medium as recited in claim 14 , wherein the UI further includes an option to select multiple time series.
17. The non-transitory machine-readable storage medium as recited in claim 14 , wherein the machine further performs operations comprising:
providing an option in the UI for entering additional metadata for searching the log messages associated with the first time series.
18. The non-transitory machine-readable storage medium as recited in claim 14 , wherein the machine further performs operations comprising:
providing an option in the UI for entering additional log search terms for searching the log messages associated with the first time series.
19. The method as recited in claim 1 , further comprising:
providing a first option in the UI for selecting a field of a plurality of metadata fields; and
providing a second option in the UI for selecting a field value of a plurality of field values associated with the field, the field and the field value to be used for searching the log messages associated with the first time series.