Data model selection and application based on data sources
Embodiments include generating data models that may give semantic meaning for unstructured or structured data that may include data generated and/or received by search engines, including a time series engine. A method includes generating a data model for data stored in a repository. Generating the data model includes generating an initial query string, executing the initial query string on the data, generating an initial result set based on the initial query string being executed on the data, determining one or more candidate fields from one or results of the initial result set, generating a candidate data model based on the one or more candidate fields, iteratively modifying the candidate data model until the candidate data model models the data, and using the candidate data model as the data model.
1 . A computer-implemented method, comprising:
generating a result set from data stored in a data repository using a query string, wherein the data stored in the data repository comprises a plurality of time-stamped, searchable events including a portion of unstructured raw machine data reflecting activity in an information technology environment;
determining a plurality of candidate data model object fields based upon the result set;
generating groupings for the plurality of candidate data model object fields based upon at least one field commonality among the plurality of candidate data model object fields of the result set; and
generating, using the result set from the query string, a data model that defines semantic meaning for at least a portion of the data stored in the data repository, that is maintained in an unmodified form, based upon the groupings for the plurality of candidate data model object fields determined based on the result set corresponding with the query string.
2 . The computer-implemented method of claim 1 , wherein determining the plurality of candidate data model object fields comprises causing a graphical user interface to be displayed that includes a list of candidate data model object fields, wherein the plurality of candidate data model object fields are selected from the list of candidate data model object fields via the graphical user interface.
3 . The computer-implemented method of claim 1 , wherein determining the plurality of candidate data model object fields comprises identifying a plurality of candidate field names in an unstructured data record corresponding to the result set.
4 . The computer-implemented method of claim 1 , wherein generating the data model based upon the groupings for the plurality of candidate data model object fields comprises iteratively modifying the groupings or the plurality of candidate data model object fields.
5 . The computer-implemented method of claim 1 , wherein determining the groupings of the plurality of candidate data model object fields comprises identifying records in the result set having at least one common field.
6 . The computer-implemented method of claim 1 , wherein determining the groupings of the plurality of candidate data model object fields comprises mapping at least one common field in the plurality of candidate data model object fields to a field in the data model.
7 . The computer-implemented method of claim 1 , further comprising identifying a parent-child relationship between records in the result set having one or more fields in common, wherein the parent-child relationship between the records is mapped to a field in the data model.
8 . The computer-implemented method of claim 1 , wherein determining the groupings of the plurality of candidate data model object fields comprises identifying a plurality of records associated with a single information technology event in the result set.
9 . The computer-implemented method of claim 1 , wherein generating the result set comprises identifying results based upon a report template and determining the plurality of candidate data model object fields based upon the result set comprises mapping the result set for a plurality of fields specified in the report template.
10 . One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps of:
generating a result set from data stored in a data repository using a query string, wherein the data stored in the data repository comprises a plurality of time-stamped, searchable events including a portion of unstructured raw machine data reflecting activity in an information technology environment;
determining a plurality of candidate data model object fields based upon the result set;
generating groupings for the plurality of candidate data model object fields based upon at least one field commonality among the plurality of candidate data model object fields of the result set; and
generating, using the result set from the query string, a data model that defines semantic meaning for at least a portion of the data stored in the data repository, that is maintained in an unmodified form, based upon the groupings for the plurality of candidate data model object fields determined based on the result set corresponding with the query string.
11 . The one or more non-transitory computer readable media of claim 10 , wherein determining the plurality of candidate data model object fields comprises causing a graphical user interface to be displayed that includes a list of candidate data model object fields, wherein the plurality of candidate data model object fields are selected from the list of candidate data model object fields via the graphical user interface.
12 . The one or more non-transitory computer readable media of claim 10 , wherein determining the plurality of candidate data model object fields comprises identifying a plurality of candidate field names in an unstructured data record corresponding to the result set.
13 . The one or more non-transitory computer readable media of claim 10 , wherein determining the groupings of the plurality of candidate data model object fields comprises identifying records in the result set having at least one common field.
14 . The one or more non-transitory computer readable media of claim 10 , wherein determining the groupings of the plurality of candidate data model object fields comprises mapping at least one common field in the plurality of candidate data model object fields to a field in the data model.
15 . The one or more non-transitory computer readable media of claim 10 , further comprising identifying a parent-child relationship between records in the result set having one or more fields in common, wherein the parent-child relationship between the records is mapped to a field in the data model.
16 . The one or more non-transitory computer readable media of claim 10 , wherein generating the result set comprises identifying results based upon a report template and determining the plurality of candidate data model object fields based upon the result set comprises mapping the result set for a plurality of fields specified in the report template.
17 . The one or more non-transitory computer readable media of claim 10 , further comprising generating a pivot report based upon the result set and the data model, wherein the data model specifies a cell calculation extracted from a report template.
18 . A computer system, comprising:
one or more memories; and
one or more processors for:
generating a result set from data stored in a data repository using a query string, wherein the data stored in the data repository comprises a plurality of time-stamped, searchable events including a portion of unstructured raw machine data reflecting activity in an information technology environment;
determining a plurality of candidate data model object fields based upon the result set;
generating groupings for the plurality of candidate data model object fields based upon at least one field commonality among the plurality of candidate data model object fields of the result set; and
generating, using the result set from the query string, a data model that defines semantic meaning for at least a portion of the data stored in the data repository, that is maintained in an unmodified form, based upon the groupings for the plurality of candidate data model object fields determined based on the result set corresponding with the query string.