Processing of performance data and raw log data from an information technology environment using search criterion input via a graphical user interface
The disclosed system and method acquire and store performance measurements relating to performance of a component in an information technology (IT) environment and log data produced by the IT environment, in association with corresponding time stamps. The disclosed system and method correlate at least one of the performance measurements with at least one of the portions of log data.
1. A method comprising:
acquiring a plurality of performance measurements for a performance metric associated with at least one hardware or software component of an information technology (IT) environment;
acquiring a plurality of portions of raw log data from at least one log file, the portions of raw log data representing activity of at least one hardware or software component of the IT environment;
storing the acquired performance measurements in a first format;
storing the acquired portions of raw log data from the at least one log file in a second format that is different from the first format;
obtaining a search criterion via a graphical user interface element;
generating a search query including the search criterion input via the graphical user interface element; and
correlating, by the computer, at least one of the stored performance measurements with at least one of the stored portions of raw log data from the at least one log file, based on a correlation criterion, wherein said correlating includes
in response to the search query including the search criterion input via the graphical user interface element, applying the search query to the stored performance measurements in the first format and the stored portions of log data in the second format that is different from the first format,
causing display, via a graphical user interface, of a performance measurement that satisfies the search criterion, and
causing display, via the graphical user interface, of a portion of raw log data, from the at least one log file, that satisfies the search criterion.
2. The method of claim 1 , wherein the search criterion is a time-based criterion.
3. The method of claim 1 , wherein the search criterion is not a time-based criterion.
4. The method of claim 1 , wherein the search criterion is a user-specified search criterion.
5. The method of claim 1 , wherein the search criterion is a user-specified search criterion that is not a time-based criterion.
6. The method of claim 1 , wherein said correlating comprises causing concurrent display of the performance measurement that satisfies the search criterion and the portion of raw log data that satisfies the search criterion.
7. The method of claim 1 , wherein the search criterion relates to a machine in the IT environment.
8. The method of claim 1 , wherein the search criterion is obtained via a user input to the graphical user interface element that enables a user to input the search criterion but does not enable input of a complete search query.
9. The method of claim 1 , wherein the search criterion is obtained via a user's selection from a drop-down list.
10. The method of claim 1 , wherein the search criterion relates to a machine in the IT environment and is specified by a user's selection from a drop-down list of machines included in the IT environment.
11. The method of claim 1 , wherein
the storing comprises storing each of the acquired performance measurements and each of the acquired portions of raw log data with a time-stamp in a time-series data store;
the correlating further comprises identifying at least one of the stored performance measurements and at least one of the stored portions of raw log data that have time stamps that satisfy a user-specified time criterion.
12. The method of claim 1 , wherein the performance measurements have been determined by direct measurement of a hardware or software component in the IT environment.
13. The method of claim 1 , wherein the plurality of portions of raw log data are derived from a text-based log file.
14. The method of claim 1 , wherein the plurality of performance measurements are not from a log file and are acquired independently of the plurality of portions of raw log data.
15. The method of claim 1 , wherein the plurality of performance measurements are acquired independently of the plurality of portions of log data, and wherein the performance measurements have been determined by direct measurement of a hardware or software component in the IT environment and the plurality of portions of raw log data are from a text-based log file.
16. The method of claim 1 , wherein the plurality of performance measurements are acquired independently of the plurality of portions of log data by direct measurement of a hardware or software component in the IT environment, and the plurality of portions of raw log data are acquired independently of the performance measurements.
17. The method of claim 1 , wherein acquiring the plurality of performance measurements comprises acquiring, via an application programming interface (API), the plurality of performance measurements from a third-party software application that collects the performance measurements.
18. The method of claim 1 , wherein the performance measurements are stored in a first time-series data store, and the portions of raw log data are stored in a second time-series data store separate from the first time-series data store.
19. The method of claim 1 , wherein the performance measurements are stored in a time-series data store in the first format, and the portions of raw log data are stored in said time-series data store in the second format different from the first format.
20. The method of claim 1 , wherein the performance measurements are stored in a first time-series data store in the first format, and the portions of raw log data are stored in a second time-series data store separate from the first time-series data store in the second format different from the first format.
21. The method of claim 1 , wherein the performance measurements are stored in a first time-series data store in the first format, and the portions of raw log data are stored in a second time-series data store separate from the first time-series data store in the second format different from the first format;
wherein the search criterion comprises a user-specified value or a user-specified range of values, for a user-specified field; and
wherein the correlating includes, in response to the user-specified value or the user-specified range of values, searching the first time-series data store for performance data that satisfy the search criterion and searching the second time-series data store for log data that satisfy the search criterion.
22. The method of claim 1 , further comprising:
acquiring structure data indicative of structure characteristics of the IT environment;
storing the acquired structure data indicative of structure characteristics of the IT environment; and
correlating a performance characteristic of the IT environment with a structure characteristic of the IT environment, based on the stored performance measurements and stored structure data.
23. The method of claim 1 , further comprising:
acquiring structure data indicative of structure characteristics of the IT environment, wherein the structure data is derived from log data from the IT environment;
storing the acquired structure data indicative of structure characteristics of the IT environment; and
correlating a performance characteristic of the IT environment with a structure characteristic of the IT environment, based on the stored performance measurements and stored structure data.
24. The method of claim 1 , wherein the performance metric comprises a performance metric for at least one hardware or software resource of a computer system.
25. The method of claim 1 , wherein the performance metric comprises a performance metric for at least one virtual machine or virtual machine host.
26. The method of claim 1 , wherein the performance metric comprises a performance metric for a virtual machine cluster.
27. A non-transitory machine-readable storage medium for use in a processing system of a data intake and query system, the non-transitory machine-readable storage medium storing instructions, an execution of which in the processing system causes the processing system to perform operations comprising:
acquiring a plurality of performance measurements for a performance metric associated with at least one hardware or software component of an information technology (IT) environment;
acquiring a plurality of portions of raw log data from at least one log file, the portions of raw log data representing activity of at least one hardware or software component of the IT environment;
storing the acquired performance measurements in a first format;
storing the acquired portions of raw log data from the at least one log file in a second format that is different from the first format;
obtaining a search criterion via a graphical user interface element;
generating a search query including the search criterion input via the graphical user interface element; and
correlating at least one of the stored performance measurements with at least one of the stored portions of raw log data from the at least one log file, based on a correlation criterion, wherein said correlating includes
in response to the search query including the search criterion input via the graphical user interface element, applying the search query to the stored performance measurements in the first format and the stored portions of log data in the second format that is different from the first format,
causing display, via a graphical user interface, of a performance measurement that satisfies the search criterion, and
causing display, via the graphical user interface, of a portion of raw log data, from the at least one log file, that satisfies the search criterion.
28. A system comprising:
a communication device through which to communicate on a computer network;
and
at least one processor operatively coupled to the communication device and configured to perform operations including acquiring a plurality of performance measurements for a performance metric associated with at least one hardware or software component of an information technology (IT) environment;
acquiring a plurality of portions of raw log data from at least one log file, the portions of raw log data representing activity of at least one hardware or software component of the IT environment;
storing the acquired performance measurements in a first format;
storing the acquired portions of raw log data from the at least one log file in a second format that is different from the first format;
obtaining a search criterion via a graphical user interface element;
generating a search query including the search criterion input via the graphical user interface element; and
correlating at least one of the stored performance measurements with at least one of the stored portions of raw log data from the at least one log file, based on a correlation criterion, wherein said correlating includes
in response to the search query including the search criterion input via the graphical user interface element, applying the search query to the stored performance measurements in the first format and the stored portions of log data in the second format that is different from the first format,
causing display, via a graphical user interface, of a performance measurement that satisfies the search criterion, and
causing display, via the graphical user interface, of a portion of raw log data, from the at least one log file, that satisfies the search criterion.
29. The system of claim 28 , wherein the search criterion includes a time-based criterion and relates to a machine in the IT environment.