APPLYING DATA-DETERMINANT QUERY TERMS TO DATA RECORDS WITH DIFFERENT FORMATS
Systems and methods are disclosed for flexibly applying a query term to heterogeneous data. A query system can receive a query that includes a data-determinant query term. As the system executes the query it can generate interim search results. As the system query processes the interim search results based on the query, it can apply the data-determinant query term to records of the interims search results based on the structure of the records.
1 .- 20 . (canceled)
21 . A method, comprising:
receiving a plurality of data records, the plurality of data records having a plurality of formats;
identifying a data-determinant query term of a query to be applied to the plurality of data records;
determining a first data record of the plurality of data records has a first format of the plurality of formats;
applying the data-determinant query term to a first field of the first data record based on determining the first data record has the first format;
determining a second data record of the plurality of data records has a second format of the plurality of formats, wherein the second format is different from the first format;
applying the data-determinant query term to a second field of the second data record based on determining the second data record has the second format; and
generating a set of search results based on applying the data-determinant query term to the first field of the first data record and the second field of the second data record.
22 . The method of claim 21 , wherein the first data record includes a plurality of field identifiers for fields of the first data record, wherein the second data record includes a plurality of field identifiers for fields of the second data record, wherein the plurality of field identifiers are distinct.
23 . The method of claim 21 , wherein the first data record includes a first quantity of fields and the second data record includes a second quantity of fields, wherein the first quantity of fields and the second quantity of fields are distinct.
24 . The method of claim 21 , wherein the first data record includes a single field value for the first field and the second data record includes a data structure comprising a plurality of properties and property values for the second field.
25 . The method of claim 21 , wherein applying the data-determinant query term to the first field of the first data record includes determining how to apply the data-determinant query term based on metadata associated with the first data record.
26 . The method of claim 21 , wherein applying the data-determinant query term to the first field of the first data record includes determining how to apply the data-determinant query term based on metadata associated with the first data record, including a host, source, source-type, or index of data obtained from a data source that corresponds to the first data record.
27 . The method of claim 21 , wherein identifying the data-determinant query term comprises identifying the data-determinant query term based on an identification of a query command that precedes the data-determinant query term.
28 . The method of claim 21 , wherein identifying the data-determinant query term comprises identifying the data-determinant query term based on a determination that the plurality of data records includes heterogeneous data.
29 . The method of claim 21 , wherein identifying the data-determinant query term comprises identifying the data-determinant query term based on a determination that the first data record has the first format of the plurality of formats and the second data record has the second format of the plurality of formats, wherein the second format is different from the first format.
30 . The method of claim 21 , wherein identifying the data-determinant query term includes a determination that the first data record includes a first set of fields with corresponding field values and the second data record includes a second set of fields with corresponding field values, wherein the first set of fields and the second set of fields are distinct.
31 . The method of claim 21 , wherein applying the data-determinant query term to the first field of the first data record comprises:
determining that the data-determinant query term matches a field identifier for the first field of the first data record; and
applying a search command of the data-determinant query term to a field value of the first field of the first data record.
32 . The method of claim 21 , wherein applying the data-determinant query term to the first field of the first data record comprises determining that a first portion of the data-determinant query term matches a field identifier for the first field of the first data record, and wherein applying the data-determinant query term to the second field of the second data record comprises determining that a second portion of the data-determinant query term matches a sub-field identifier for a sub-field of the second field of the first data record.
33 . The method of claim 21 , wherein applying the data-determinant query term to the second field of the second data record comprises:
determining that a first portion of the data-determinant query term matches a data structure identifier of the second data record and a second portion of the data-determinant query term matches a property identifier for a property of a data structure corresponding to the data structure identifier; and
applying a search command of the data-determinant query term to a property value of the property.
34 . The method of claim 21 , wherein applying the data-determinant query term to the second field of the second data record comprises:
determining that the data-determinant query term does not match a field identifier of the second data record;
identifying a dataset associated with the second data record based on the data-determinant query term;
determining a content value for the second data record based on a lookup operation using the dataset associated with the second data record; and
applying a search command of the data-determinant query term to the content value.
35 . The method of claim 21 , wherein applying the data-determinant query term to the second field of the second data record comprises:
determining that the data-determinant query term does not match a field identifier of the second data record;
identifying a dataset associated with the second data record based on the data-determinant query term;
determining a content value for the second data record based on a lookup operation using the dataset associated with the second data record; and
applying a search command of the data-determinant query term to the content value,
wherein the plurality of data records includes a third data record that corresponds to the second data record of the plurality of data records, and wherein the third data record includes a data structure identifier and a property identifier that correspond to the data-determinant query term and a property value that corresponds to the content value.
36 . The method of claim 21 , wherein applying the data-determinant query term to the second field of the second data record comprises:
determining that the data-determinant query term does not match a field identifier of the second data record;
identifying a dataset associated with the second data record based on the data-determinant query term;
determining a content value for the second data record based on a lookup operation using the dataset associated with the second data record; and
applying a search command of the data-determinant query term to the content value,
wherein the plurality of data records includes a third data record that corresponds to the second data record of the plurality of data records, and wherein the third data record includes a field identifier that corresponds to the data-determinant query term and a field value that corresponds to the content value.
37 . The method of claim 21 , wherein applying the data-determinant query term to the second field of the second data record comprises:
determining that the data-determinant query term does not match a field identifier of the second data record;
identifying a regular expression rule associated with the second data record based on the data-determinant query term;
determining a first field value of the second field of the second data record based on the regular expression rule; and
applying a search command of the data-determinant query term to the first field value,
wherein the plurality of data records includes a third data record that corresponds to the second data record of the plurality of data records, and wherein the third data record includes a field identifier that corresponds to the data-determinant query term and a second field value that corresponds to the first field value.
38 . The method of claim 21 , wherein applying the data-determinant query term to the second field of the second data record comprises:
determining that the data-determinant query term does not match any field identifier of the second data record;
identifying a regular expression rule associated with the second data record based on the data-determinant query term;
determining a first field value of the second field of the second data record based on the regular expression rule;
generating a field identifier for the second data record, the generated field identifier corresponding to the second field;
adding the first field value to the second data record in association with the generated field identifier; and
applying a search command of the data-determinant query term to the first field value.
39 . A computing system of a data intake and query system, the computing system comprising:
memory; and
one or more processing devices coupled to the memory and configured to:
receive a plurality of data records, the plurality of data records having a plurality of formats;
identify a data-determinant query term of a query to be applied to the plurality of data records;
determine a first data record of the plurality of data records has a first format of the plurality of formats;
apply the data-determinant query term to a first field of the first data record based on determining the first data record has the first format;
determine a second data record of the plurality of data records has a second format of the plurality of formats, wherein the second format is different from the first format;
apply the data-determinant query term to a second field of the second data record based on determining the second data record has the second format; and
generate a set of search results based on applying the data-determinant query term to the first field of the first data record and the second field of the second data record.
40 . Non-transitory computer readable media comprising computer-executable instructions that, when executed by a computing system of a query system, cause the computing system to:
receive a plurality of data records, the plurality of data records having a plurality of formats;
identify a data-determinant query term of a query to be applied to the plurality of data records;
determine a first data record of the plurality of data records has a first format of the plurality of formats;
apply the data-determinant query term to a first field of the first data record based on determining the first data record has the first format;
determine a second data record of the plurality of data records has a second format of the plurality of formats, wherein the second format is different from the first format;
apply the data-determinant query term to a second field of the second data record based on determining the second data record has the second format; and
generate a set of search results based on applying the data-determinant query term to the first field of the first data record and the second field of the second data record.