Automatic entity definitions
Methods are disclosed to take advantage of the early collection of machine data from a new or changed entity in a computing environment in order to update the definitional information about entities used by a service monitoring system. In some embodiments, the process undertaken to recognize new or changed entities in an IT environment from collected machine data may be informed by the expertise of a particular subject matter area by installing that intelligence in a codified form packaged as a domain add-on to the service monitoring system.
1. A method comprising:
receiving and storing machine data from a plurality of data sources over a period of time;
applying data selection criteria to the machine data to select particular machine data identifying an entity, the entity having information generated by or about the entity among the machine data, wherein the entity hosts at least one of the plurality of data sources and generates at least a portion of the machine data as part of performing a service, wherein the service is provided by one or more entities and has a service definition specifying entity definitions corresponding to the one or more entities providing the service, wherein applying the data selection criteria to the machine data results in discovering undefined entities by selecting the particular machine data from multiple sources associated with multiple source types, the undefined entities comprising entities without a corresponding entity definition being stored in computer storage;
determining, based on the particular machine data, whether the identified entity has a corresponding entity definition physically represented in computer storage;
in response to determining that the identified entity does not have the corresponding entity definition, deriving content descriptive of the entity from the information generated by or about the entity, wherein deriving the content descriptive of the entity includes executing a second search query to produce a search result set having an entry corresponding to the entity, the entry having one or more data items, and a position of at least one of the data items in the entry indicating an element of the stored entity definition; and
generating a stored entity definition based at least in part on the derived content, the stored entity definition comprising information to identify the entity, to associate the entity with the service definition of the service, and to associate the entity with portions of the machine data, wherein the stored entity definition is physically represented in the computer storage;
wherein the entity performs the service and the service has a key performance indicator (KPI) associated with a first search query of the machine data that derives a value indicative of the performance of the service at a point in time or during a period of time, the value derived from machine data identified in the entity definitions for the one or more entities that provide the service, wherein the entity definitions comprise the stored entity definition;
wherein the method is performed by a computer system comprising one or more processors.
2. The method of claim 1 , wherein the information generated by or about the entity comprises data generated by the entity.
3. The method of claim 1 , wherein the information generated by or about the entity comprises data generated by a machine other than the entity.
4. The method of claim 1 , wherein the information generated by or about the entity comprises data generated by the entity and a machine other than the entity.
5. The method of claim 1 , wherein the content descriptive of the entity includes an identifier.
6. The method of claim 1 , wherein the content descriptive of the entity includes an alias.
7. The method of claim 1 , wherein the content descriptive of the entity includes at least one from among a network address, a host name, and an identifier number.
8. The method of claim 1 , wherein the deriving content descriptive of the entity includes extracting information from machine data from multiple sources.
9. The method of claim 1 , wherein the identifying an entity having information generated by or about the entity among the machine data comprises accessing the machine data using a late-binding schema.
10. The method of claim 1 , wherein the storing machine data comprises storing machine data as timestamped events.
11. The method of claim 1 , wherein the storing machine data comprises storing machine data as timestamped events each having a segment of raw machine data.
12. The method of claim 1 , wherein deriving the content descriptive of the entity includes executing the second search query to produce the search result set in tabular form.
13. The method of claim 1 , wherein generating the stored entity definition includes processing the search result set as an input.
14. The method of claim 1 , further comprising causing display of a graphical user interface (GUI) having information from the stored entity definition including at least one from among a name, an alias, and an information field.
15. The method of claim 1 , further comprising:
determining whether the stored entity definition satisfies filter criteria for identifying entities associated with the service.
16. The method of claim 1 , further comprising:
determining whether the stored entity definition satisfies filter criteria for identifying entities associated with the service; and
associating the stored entity definition with the service definition for the service in accordance with said determining.
17. The method of claim 1 , wherein the identifying an entity is performed automatically in accordance with a frequency or schedule.
18. A system comprising:
a memory; and
a processing device coupled with the memory to:
receive and store machine data from a plurality of data sources over a period of time;
apply data selection criteria to the machine data to select particular machine data identifying an entity, the entity having information generated by or about the entity among the machine data, wherein the entity hosts at least one of the plurality of data sources and generates at least a portion of the machine data as part of performing a service, wherein the service is provided by one or more entities and has a service definition specifying entity definitions corresponding to the one or more entities providing the service, wherein applying the data selection criteria to the machine data results in discovering undefined entities by selecting the particular machine data from multiple sources associated with multiple source types, the undefined entities comprising entities without a corresponding entity definition being stored in computer storage;
determine, based on the particular machine data, whether the identified entity has a corresponding entity definition physically represented in computer storage;
in response to determining that the identified entity does not have the corresponding entity definition, derive content descriptive of the entity from the information generated by or about the entity, wherein deriving the content descriptive of the entity includes executing a second search query to produce a search result set having an entry corresponding to the entity, the entry having one or more data items, and a position of at least one of the data items in the entry indicating an element of the stored entity definition; and
generate a stored entity definition based at least in part on the derived content, the stored entity definition comprising information to identify the entity, to associate the entity with the service definition of the service, and to associate the entity with portions of the machine data, wherein the stored entity definition is physically represented in the computer storage;
wherein the entity performs the service and the service has a key performance indicator (KPI) associated with a first search query of the machine data that derives a value indicative of the performance of the service at a point in time or during a period of time, the value derived from machine data identified in the entity definitions for the one or more entities that provide the service, wherein the entity definitions comprise the stored entity definition.
19. The system of claim 18 , wherein to derive content descriptive of the entity includes extracting information from machine data from multiple sources.
20. The system of claim 18 , wherein to identify an entity having information generated by or about the entity among the machine data comprises accessing the machine data using a late-binding schema.
21. The system of claim 18 , wherein to store machine data comprises storing machine data as timestamped events each having a segment of raw machine data.
22. The system of claim 18 , further comprising to:
make a determination whether the stored entity definition satisfies filter criteria for identifying entities associated with the service; and
associate the stored entity definition with the service definition for the service in accordance with said determination.
23. The system of claim 18 , wherein deriving the content descriptive of the entity includes executing the second search query to produce the search result set in tabular form.
24. The system of claim 18 , wherein generating the stored entity definition includes processing the search result set as an input.
25. A non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by one or more processing devices, cause the processing device to perform operations comprising:
receiving and storing machine data from a plurality of data sources over a period of time;
applying data selection criteria to the machine data to select particular machine data identifying an entity, the entity having information generated by or about the entity among the machine data, wherein the entity hosts at least one of the plurality of data sources and generates at least a portion of the machine data as part of performing a service, wherein the service is provided by one or more entities and has a service definition specifying entity definitions corresponding to the one or more entities providing the service, wherein applying the data selection criteria to the machine data results in discovering undefined entities by selecting the particular machine data from multiple sources associated with multiple source types, the undefined entities comprising entities without a corresponding entity definition being stored in computer storage;
determining, based on the particular machine data, whether the identified entity has a corresponding entity definition physically represented in computer storage;
in response to determining that the identified entity does not have the corresponding entity definition, deriving content descriptive of the entity from the information generated by or about the entity, wherein deriving the content descriptive of the entity includes executing a second search query to produce a search result set having an entry corresponding to the entity, the entry having one or more data items, and a position of at least one of the data items in the entry indicating an element of the stored entity definition; and
generating a stored entity definition based at least in part on the derived content, the stored entity definition comprising information to identify the entity, to associate the entity with the service definition of the service, and to associate the entity with portions of the machine data, wherein the stored entity definition is physically represented in the computer storage;
wherein the entity performs the service and the service has a key performance indicator (KPI) associated with a first search query of the machine data that derives a value indicative of the performance of the service at a point in time or during a period of time, the value derived from machine data identified in the entity definitions for the one or more entities that provide the service, wherein the entity definitions comprise the stored entity definition.
26. The non-transitory computer readable storage medium of claim 25 , wherein the deriving content descriptive of the entity includes extracting information from machine data from multiple sources.
27. The non-transitory computer readable storage medium of claim 25 , wherein the identifying an entity having information generated by or about the entity among the machine data comprises accessing the machine data using a late-binding schema.
28. The non-transitory computer readable storage medium of claim 25 , further comprising:
determining whether the stored entity definition satisfies filter criteria for identifying entities associated with the service; and
associating the stored entity definition with the service definition for the service in accordance with said determining.
29. The non-transitory computer readable storage medium of claim 25 , wherein deriving the content descriptive of the entity includes executing the second search query to produce the search result set in tabular form.
30. The non-transitory computer readable storage medium of claim 25 , wherein generating the stored entity definition includes processing the search result set as an input.