Model-based systems and methods for monitoring computing resource performance
View Patent ↗Modeling systems and methods for constructing one or more models of a computing system using collected data. Certain model-based systems build topology models and/or model instances by transforming collected performance data into a collection-location independent form. In certain examples, systems include at least one agent for collecting performance data from monitored resource(s), canonical data transform (CDT) configurations, and a data transformation module for performing data transform operation(s) on the performance data based on at least one CDT configuration. The data transform operation may include generating and/or updating a topology model, assigning metrics to model object(s), updating properties of model object(s), creating associations between existing model objects, or the like. Certain systems and methods also allow for a single piece of data to be processed by multiple models or for pieces of data collected from different locations to be matched and/or associated with the same model object.
1. A method for dynamically modeling and monitoring performance data in a computing environment, the method comprising:
by a computer system comprising computer hardware:
receiving performance data collected by at least one agent module from multiple applications executing in multiple servers in a computing environment, wherein the performance data comprises a first data tree having a plurality of different types of data nodes, wherein the different types of data nodes are associated with different time series metrics about the multiple servers and the multiple applications executing in the multiple servers;
automatically creating a topological arrangement of the plurality of different types of data nodes in the first data tree by automatically creating a second data tree of configuration nodes, wherein the second data tree of configuration notes are configured to store the performance data in a topological manner that represents topological relationships of the different types of data nodes in the first data tree,
wherein the second data tree of configuration nodes are automatically associated with the first data tree of different types of data nodes such that the configuration nodes represent relationships between the plurality of different types of data nodes in the first data tree, and
automatically creating the second data tree by accessing a first model template of a plurality of model templates wherein the model template comprises a topology of configuration nodes representing the expected structure of the first data tree, wherein a structure of the first model template comprises a second tree structure having a plurality of configuration nodes, wherein the second tree structure initially reflects a pre-defined structure of associated with the type of performance data and is independent of where the performance data was collected by the at least one agent module;
automatically comparing properties of each of the plurality of different types of data nodes of the first data tree with properties of the configuration nodes of the second data tree to identify matches between at least a portion of the plurality of different types of data nodes and a portion of the configuration nodes;
automatically generating a modified model of the second data tree that comprises the topology of configuration nodes in the second data tree that match the structure of the plurality of different types of data nodes in the first data tree, wherein the modified model of the second data tree comprises a plurality of interconnected topology objects representing the multiple applications executing in the multiple servers of the computing environment;
storing the modified model in a data repository; and
storing monitored performance data associated with the multiple applications that continue to execute in the multiple servers of the computer environment within the topology of configuration nodes of the second data tree, wherein the configuration nodes of the second data tree store the different time series metrics about the multiple servers and the multiple applications executing in the multiple servers; and
displaying the topological arrangement of the monitored performance data based on the topology of configuration nodes of the second data tree.
2. The method of claim 1 , wherein said automatically generating the modified model comprises for each one of the different types of data nodes in the first data tree:
identifying a corresponding configuration node, if any, of the plurality of configuration nodes; and
performing a data transform operation on a data node associated with the corresponding configuration node, the data transform operation being based on the definition associated with the corresponding configuration node.
3. The method of claim 2 , wherein the data transform operation comprises creating a new topology object.
4. The method of claim 2 , wherein the data transform operation comprises associating metrics to an existing topology object.
5. The method of claim 2 , wherein the data transform operation comprises creating an association between two existing topology objects.
6. The method of claim 1 , additionally comprising:
accessing a second model template of the plurality of model templates different from said first model template;
generating a second modified model having a plurality of second topology objects based at least in part on the performance data and the second model template; and
storing the second modified model in the data repository.
7. The method of claim 6 , additionally comprising receiving user input to generate a third model based on the plurality of topology objects of the first modified model and the plurality of second topology objects of the second modified model.
8. The method of claim 1 , wherein the at least one agent module comprises a first agent module and a second agent module, and wherein said generating comprises matching performance data collected by the first agent module with performance data collected by the second data module.
9. The method of claim 8 , wherein said generating comprises associating matching data from the first and second agent modules with a single topology object of the plurality of topology objects of the modified model.
10. The method of claim 1 , wherein the modified model comprises a model instance.
11. The method of claim 10 , additionally comprising applying one or more rules to data of the model instance.
12. The method of claim 1 , additionally comprising modifying a second model based on changes made while generating the modified model.
13. The method of claim 1 , additionally comprising generating a user notification when said generating comprises modifying a non-identity property of one of the plurality of topology objects.
14. A system for monitoring data in a computing environment, the system comprising:
an agent module executing on a computing device and configured to obtain from at least one computing resource raw data regarding a performance of multiple applications executing in multiple servers in a computing environment, the raw data comprising a first data tree having a plurality of different types of data nodes, wherein the different types of data nodes are associated with different time series metrics about the multiple servers and the multiple applications executing in the multiple servers;
a database storing a plurality of data transform configurations each representing a standardized data structure having a plurality of configuration nodes, wherein each of the configuration nodes defines at least one action to perform on the raw data, and wherein a selected one of the plurality of data transform configurations comprises a structure indicative of an anticipated structure of the raw data irrespective of where the raw data was collected by the agent module; and
a data transformation module implemented in one or more computer processors, the data transformation module configured to,
receive the raw data collected by the agent module,
automatically compare each of the plurality of different types of data nodes of the first data tree associated with the raw data with at least one of the plurality of configuration nodes of the second data tree the selected data transform configuration, and
transform the raw data into a second data tree that matches the first data tree such that the configuration nodes of the second data tree represent relationships between the plurality of different types of data nodes in the first data tree, wherein the configuration notes are configured to store the different time series metrics about the multiple servers and the multiple applications executing in the multiple servers data associated with the different types of data nodes in a topological manner that represents topological relationships of the different types of data nodes in the first data tree based at least on the selected data transform configuration, the topological relationships comprising a plurality of interconnected topology objects representing the multiple applications executing in multiple servers in a computing environment.
15. The system of claim 14 , wherein the data transformation module is configured to merge a new topology object with existing topology objects to form the topological relationships.
16. The system of claim 14 , further comprising at least one handler module in communication with the agent module and the data transformation module, the at least one handler module being configured to standardize the raw data for processing by the data transformation module.
17. The system of claim 16 , wherein the at least one handler module comprises an application programming interface (API).
18. The system of claim 14 , wherein the data transformation module is configured to attach metrics to at least one of the plurality of interconnected topology objects of the topology model.