Real-time measurement and system monitoring based on generated dependency graph models of system components
Machine data is collected from multiple sources of an operating environment such as an information technology system, factory floor, or the like, into a data intake and query system, in one embodiment. Metrics representative of the environment are included in or derived from the data. Users may interact with an interface to depict a representation of various metrics and interdependencies and that depiction is reflected in a computer storage model. Changes to the computer storage model based on the user interaction may also result in training of a machine learning model according to the user interaction, the machine learning model configured to determine a prediction, classification or clustering of a result of a first search query by utilizing a result of at least a second search query as input to the machine learning model.
1. A computerized method for generating a dynamic user interface utilizing machine learning techniques, the method comprising:
receiving, by logic upon execution by one or more processors, first user input that defines a first relationship and a second relationship, wherein the first relationship indicates that a first node is dependent on a second node and the second relationship indicates that the first node is also dependent on a third node, and wherein the first node, the second node, and the third node are included in a dependency graph that is displayed via the dynamic user interface,
wherein the first node represents first nodal metrics including a first nodal name, a first data source, and a first search query, and
wherein the second node represents second nodal metrics including a second data source, and a second search query, and
wherein the third node represents third nodal metrics including a third data source, and a third search query;
receiving, by the logic, second user input applying a machine learning model to results of each of the second search query and the third search query;
prior to applying the machine learning model, training, by the logic, the machine learning model using historical data of the first data source;
executing, by the logic, the second search query on data from the second data source and the third search query on data from the third data source to generate search query results;
applying, by the logic, the machine learning model to the results of each of the second search query and the third search query resulting in generation of a predicted result of the first search query; and
causing, by the logic, rendering of the dynamic user interface to include a display of the predicted result of the first search query.
2. The computerized method of claim 1 , further comprising:
receiving, by the logic, third user input corresponding to the first nodal metrics and the second nodal metrics.
3. The computerized method of claim 1 , wherein the first node represents a first measurement that is observable over time, and the second node represents a second measurement that is observable over time.
4. The computerized method of claim 1 , further comprising:
retraining, by the logic, the machine learning model on a periodic basis using updated training data each subsequent training.
5. The computerized method of claim 1 , further comprising:
causing, by the logic, the dependency graph to be rendered in a first state, wherein the dependency graph is a visual representation of a causal structure that includes the first node, the second node and the first relationship.
6. The computerized method of claim 1 , wherein the first data source includes a first source of first time series data, wherein the first time series data comprises a first sequence of data points that are associated with successive points in time.
7. The computerized method of claim 6 , wherein the first time series data is raw machine data.
8. The computerized method of claim 7 , further comprising:
parsing, by the logic, the raw machine data into a plurality of timestamped events, each timestamped event in the plurality of timestamped events comprising at least a portion of the raw machine data.
9. The computerized method of claim 1 , wherein the dynamic user interface further includes illustrative graphical representations of the observable measurements in real-time.
10. The computerized method of claim 9 , wherein the illustrative graphical representations of the observable measurements in real-time are displayed alongside predictive analytics using the machine learning techniques.
11. The computerized method of claim 10 , wherein results of the predictive analytics include the predicted result of the first search query.
12. A non-transitory computer readable storage medium having stored thereon logic, the logic being executable by one or more processors to perform operations comprising:
receiving, by the logic, first user input that defines a first relationship and a second relationship, wherein the first relationship indicates that a first node is dependent on a second node and the second relationship indicates that the first node is also dependent on a third node, and wherein the first node, the second node, and the third node are included in a dependency graph that is displayed via a dynamic user interface,
wherein the first node represents first nodal metrics including a first nodal name, a first data source, and a first search query, and
wherein the second node represents second nodal metrics including a second data source, and a second search query, and
wherein the third node represents third nodal metrics including a third data source, and a third search query;
receiving, by the logic, second user input applying a machine learning model to results of each of the second search query and the third search query;
prior to applying the machine learning model, training, by the logic, the machine learning model using historical data of the first data source;
executing, by the logic, the second search query on data from the second data source and the third search query on data from the third data source to generate search query results;
applying, by the logic, the machine learning model to the results of each of the second search query and the third search query resulting in generation of a predicted result of the first search query; and
causing, by the logic, rendering of the dynamic user interface to include a display of the predicted result of the first search query.
13. The non-transitory computer readable storage medium of claim 12 , wherein the logic being executable by the one or more processors to perform further operations comprising:
receiving, by the logic, third user input corresponding to the first nodal metrics and the second nodal metrics.
14. The non-transitory computer readable storage medium of claim 12 , wherein the first node represents a first measurement that is observable over time, and the second node represents a second measurement that is observable over time.
15. The non-transitory computer readable storage medium of claim 12 , wherein the logic being executable by the one or more processors to perform further operations comprising:
retraining, by the logic, the machine learning model on a periodic basis using updated training data each subsequent training.
16. The non-transitory computer readable storage medium of claim 12 , wherein the logic being executable by the one or more processors to perform further operations comprising:
causing, by the logic, the dependency graph to be rendered in a first state, wherein the dependency graph is a visual representation of a causal structure that includes the first node, the second node and the first relationship.
17. The non-transitory computer readable storage medium of claim 9 , wherein the first data source includes a first source of first time series data, wherein the first time series data comprises a first sequence of data points that are associated with successive points in time.
18. The non-transitory computer readable storage medium of claim 17 , wherein the first time series data is raw machine data.
19. The non-transitory computer readable storage medium of claim 18 , wherein the logic being executable by the one or more processors to perform further operations comprising:
parsing, by the logic, the raw machine data into a plurality of timestamped events, each timestamped event in the plurality of timestamped events comprising at least a portion of the raw machine data.
20. The non-transitory computer readable storage medium of claim 12 , wherein the dynamic user interface further includes illustrative graphical representations of the observable measurements in real-time.
21. The non-transitory computer readable storage medium of claim 20 , wherein the illustrative graphical representations of the observable measurements in real-time are displayed alongside predictive analytics using the machine learning techniques.
22. The non-transitory computer readable storage medium of claim 21 , wherein results of the predictive analytics include the predicted result of the first search query.
23. A system comprising:
a memory to store executable logic; and
a processing device coupled with the memory, wherein the logic, when executed by the processing device, cause operations including:
receiving, by the logic, first user input that defines a first relationship and a second relationship, wherein the first relationship indicates that a first node is dependent on a second node and the second relationship indicates that the first node is also dependent on a third node, and wherein the first node, the second node, and the third node are included in a dependency graph that is displayed via a dynamic user interface,
wherein the first node represents first nodal metrics including a first nodal name, a first data source, and a first search query, and
wherein the second node represents second nodal metrics including a second data source, and a second search query, and
wherein the third node represents third nodal metrics including a third data source, and a third search query;
receiving, by the logic, second user input applying a machine learning model to results of each of the second search query and the third search query;
prior to applying the machine learning model, training, by the logic, the machine learning model using historical data of the first data source;
executing, by the logic, the second search query on data from the second data source and the third search query on data from the third data source to generate search query results;
applying, by the logic, the machine learning model to the results of each of the second search query and the third search query resulting in generation of a predicted result of the first search query; and
causing, by the logic, rendering of the dynamic user interface to include a display of the predicted result of the first search query.
24. The system of claim 23 , wherein the logic being executable by the processing device to perform further operations comprising:
receiving, by the logic, third user input corresponding to the first nodal metrics and the second nodal metrics.
25. The system of claim 23 , wherein the first node represents a first measurement that is observable over time, and the second node represents a second measurement that is observable over time.
26. The system of claim 23 , wherein the logic being executable by the processing device to perform further operations comprising:
retraining, by the logic, the machine learning model on a periodic basis using updated training data each subsequent training.
27. The system of claim 23 , wherein the logic being executable by the processing device to perform further operations comprising:
causing, by the logic, the dependency graph to be rendered in a first state, wherein the dependency graph is a visual representation of a causal structure that includes the first node, the second node and the first relationship.
28. The system of claim 23 , wherein the first data source includes a first source of first time series data, wherein the first time series data comprises a first sequence of data points that are associated with successive points in time.
29. The system of claim 28 , wherein the first time series data is raw machine data.
30. The system of claim 29 , wherein the logic being executable by the processing device to perform further operations comprising:
parsing, by the logic, the raw machine data into a plurality of timestamped events, each timestamped event in the plurality of timestamped events comprising at least a portion of the raw machine data.
31. The system of claim 23 , wherein the dynamic user interface further includes illustrative graphical representations of the observable measurements in real-time.
32. The system of claim 31 , wherein the illustrative graphical representations of the observable measurements in real-time are displayed alongside predictive analytics using the machine learning techniques.
33. The system of claim 32 , wherein results of the predictive analytics include the predicted result of the first search query.