Model-based systems and methods for monitoring resources
View Patent ↗Modeling systems and methods for constructing one or more models of a computing system using collected data. Certain model-based systems build topology models and/or model instances by transforming collected performance data into a collection-location independent form. In certain examples, systems include at least one agent for collecting performance data from monitored resource(s), canonical data transform (CDT) configurations, and a data transformation module for performing data transform operation(s) on the performance data based on at least one CDT configuration. The data transform operation may include generating and/or updating a topology model, assigning metrics to model object(s), updating properties of model object(s), creating associations between existing model objects, or the like. Certain systems and methods also allow for a single piece of data to be processed by multiple models or for pieces of data collected from different locations to be matched and/or associated with the same model object.
1. A method for dynamically modeling and monitoring performance data in a computing environment, the method comprising:
by a computer system comprising computer hardware:
receiving performance data collected by at least one agent module from multiple applications executing in multiple servers in a computing environment, wherein the performance data includes a first data tree including a plurality of different types of data nodes, wherein the different types of data nodes are associated with different time series metrics about the multiple servers and the multiple applications executing in the multiple servers;
automatically accessing a first model template of a plurality of model templates wherein the model template includes a topology of configuration nodes representing the expected structure of the performance data;
automatically comparing properties of each of the plurality of different types of data nodes of the first data tree with properties of the configuration nodes of the first model template to identify matches between at least a portion of the plurality of different types of data nodes and a portion of the configuration nodes;
automatically generating a modified model of the first model template that includes the topology of configuration nodes in the first model template that match the structure of the plurality of different types of data nodes in the first data tree, wherein the modified model includes a plurality of interconnected topology objects representing the multiple applications executing in the multiple servers of the computing environment; and
storing the modified model in a data repository.
2. The method of claim 1 , wherein the automatically generating the modified model includes for each one of the different types of data nodes in the first data tree:
identifying a corresponding configuration node, if any, of the plurality of configuration nodes; and
performing a data transform operation on a data node associated with the corresponding configuration node, the data transform operation being based on the definition associated with the corresponding configuration node.
3. The method of claim 1 , additionally comprising:
accessing a second model template of the plurality of model templates different from the first model template;
generating a second modified model including a plurality of second interconnected topology objects based at least in part on the performance data and the second model template; and
storing the second modified model in the data repository.
4. The method of claim 3 , additionally comprising receiving user input to generate a third model based on the plurality of interconnected topology objects of the first modified model and the plurality of second interconnected topology objects of the second modified model.
5. The method of claim 1 , wherein the at least one agent module includes a first agent module and a second agent module, and wherein the generating includes matching performance data collected by the first agent module with performance data collected by the second agent module.
6. The method of claim 5 , wherein the generating includes associating matching data from the first and second agent modules with a single topology object of the plurality of interconnected topology objects of the modified model.
7. The method of claim 1 , wherein the modified model includes a model instance.
8. The method of claim 7 , additionally comprising applying one or more rules to data of the model instance.
9. The method of claim 1 , additionally comprising modifying a second model based on changes made while generating the modified model.
10. The method of claim 1 , additionally comprising generating a user notification when the generating includes modifying a non-identity property of one of the plurality of interconnected topology objects.
11. A system for monitoring data in a computing environment, the system comprising:
an agent module executing on a computing device and configured to obtain from at least one computing resource raw data regarding a performance of multiple applications executing in multiple servers in a computing environment, the raw data including a first data tree including a plurality of different types of data nodes, wherein the different types of data nodes are associated with different time series metrics about the multiple servers and the multiple applications executing in the multiple servers;
a database storing a plurality of data transform configurations, each data transform configuration including a topology of configuration nodes representing a standardized data structure of the raw data; and
a data transformation module implemented in one or more computer processors, the data transformation module configured to:
receive the raw data collected by the agent module;
automatically compare properties of each of the plurality of different types of data nodes of the first data tree with properties of the configuration nodes of a selected data transform configuration to identify matches between at least a portion of the plurality of different types of data nodes and a portion of the configuration nodes; and
transform the raw data into a modified model of the selected data transform configuration that includes the topology of the configuration nodes of the selected data transform configuration that match a structure of the plurality of different types of data nodes in the first data tree, wherein the modified model includes a plurality of interconnected topology objects representing the multiple applications executing in the multiple servers in the computing environment.
12. The system of claim 11 , wherein the data transformation module is configured to merge a new topology object with existing topology objects to form topological relationships.
13. The system of claim 11 , wherein the data transformation module is configured to attach metrics to at least one of the plurality of interconnected topology objects.
14. A method for dynamically modeling performance data in a computing environment, the method comprising:
by a computer system comprising computer hardware:
automatically receiving performance data collected by at least one agent module from multiple applications executing in multiple servers in a computing environment, the performance data associated with different time series metrics about the multiple servers and the multiple applications executing in the multiple servers, and the performance data including a performance data structure representing topological relationships between the different time series metrics about the multiple servers and the multiple applications executing in the multiple servers;
automatically accessing a first model template, wherein a structure of the first model template reflects an expected structure of the performance data;
automatically generating a modified model based at least in part on the performance data associated with matches between at least a portion of the performance data structure and at least a portion of the expected structure, the modified model including a plurality of topology objects; and
storing the modified model in a data repository.
15. The method of claim 14 , wherein the performance data includes a data tree having a plurality of data tree nodes.
16. The method of claim 15 , wherein the first model template includes a data tree structure having a plurality of configuration nodes.
17. The method of claim 16 , wherein the generating further comprises for each one of the plurality of data tree nodes:
identifying a corresponding configuration node, if any, of the plurality of configuration nodes; and
performing a data transform operation on the corresponding one of the plurality of data nodes, the data transform operation being based on at least one definition associated with the corresponding configuration node.
18. The method of claim 17 , wherein the data transform operation includes creating a new topology object.
19. The method of claim 17 , wherein the data transform operation includes associating metrics to an existing topology object.
20. The method of claim 17 , wherein the data transform operation includes creating an association between two existing topology objects.