Search integration
Search integration is described. The actions include receiving, by one or more processors, a query. The actions include identifying search results that are responsive to the query. The actions include identifying, based on the query and the search results, structured data to supplement the search results, wherein the structured data is generated from binary table data that is received and deserialized by one or more processing modules. The actions include processing the structured data and the search results. The actions include providing, for output, the processed search results and the processed structured data results.
1. A computer-implemented method comprising:
receiving a natural language query submitted by a user;
generating, using the natural language query, filter criteria to apply to structured data;
identifying search results from an unstructured dataset that are responsive to the natural language query;
identifying, based on the natural language query and the search results from the unstructured dataset, one or more attributes or metrics from a structured data dataset that correspond to the natural language query;
in response to receiving the natural language query and based on the identified one or more attributes or metrics, performing structured search operations including:
generating a plurality of database queries for processing on separate data partitions of the structured data dataset;
determining a partition attribute used to relate data among the data partitions;
using one or more values of the partition attribute to identify a particular data partition of the structured data dataset;
assigning each of the database queries to a different data partition of the structured data dataset including assigning a particular database query to the particular data partition identified using the one or more values of the partition attribute;
executing each of the database queries on the data partition to which the database query is assigned;
obtaining, from the structured data dataset, values for the identified one or more attributes or metrics from the different data partitions in response to the plurality of database queries;
combining the obtained values from the different data partitions into a result dataset; and
applying the filter criteria that was generated using the natural language query to the result dataset to obtain a filtered result dataset; and
providing, for display on a user interface:
an indication of the identified one or more attributes or metrics from the structured data dataset; and
one or more graphical representations based on (i) the values in the filtered result dataset for the identified one or more attributes or metrics and (ii) the search results from the unstructured dataset.
2. The computer-implemented method of claim 1 , comprising:
generating one or more terms of the natural language query by tokenizing the natural language query; and
assigning a meaning or interpretation to each of the one or more terms,
wherein identifying the one or more attributes or metrics is based further on the meaning or interpretation assigned to each of the one or more terms.
3. The computer-implemented method of claim 1 , comprising:
identifying one or more keywords included in the one or more attributes or metrics or the natural language query; and
providing, for display on the user interface, graphical elements identifying the one or more keywords.
4. The computer-implemented method of claim 1 , comprising:
based on the one or more attributes or metrics, selecting a type of visualization for the obtained values for the one or more attributes or metrics, wherein the one or more graphical representations include the selected type of visualization.
5. The computer-implemented method of claim 1 , wherein:
a data table in the structured data dataset is split among multiple of the data partitions, each of the multiple partitions including a portion of the data table, and
executing a database query on the assigned data partition comprises performing one or more operations on data retrieved from the portion of the data table included in the assigned data partition.
6. The computer-implemented method of claim 1 , wherein using one or more values of the partition attribute to identify a particular data partition of the structured data dataset comprises:
processing a value of the partition attribute through a hash function; and
identifying the particular data partition based on determining that the particular data partition has a partition attribute value that corresponds to the processed value of the partition attribute.
7. The method of claim 1 , wherein providing the one or more graphical representations comprises providing a preview including a graphical representation of a portion of the filtered result dataset prior to publishing the filtered result dataset to an in-memory data store.
8. The method of claim 7 , comprising:
providing, for display on the user interface, a control for publishing the filtered result dataset;
receiving data indicating user selection of the control; and
in response to receiving the data indicating the user selection of the control, publishing the filtered result dataset to the in-memory data store.
9. The method of claim 1 , comprising:
prior to receiving the natural language query submitted by the user, loading, into an in-memory data store, the structured data dataset and the unstructured dataset.
10. A system comprising:
one or more computers; and
one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving a natural language query submitted by a user;
generating, using the natural language query, filter criteria to apply to structured data;
identifying search results from an unstructured dataset that are responsive to the natural language query;
identifying, based on the natural language query and the search results from the unstructured dataset, one or more attributes or metrics from a structured data dataset that correspond to the natural language query;
in response to receiving the natural language query and based on the identified one or more attributes or metrics, performing structured search operations including:
generating a plurality of database queries for processing on separate data partitions of the structured data dataset;
determining a partition attribute used to relate data among the data partitions;
using one or more values of the partition attribute to identify a particular data partition of the structured data dataset;
assigning each of the database queries to a different data partition of the structured data dataset including assigning a particular database query to the particular data partition identified using the one or more values of the partition attribute;
executing each of the database queries on the data partition to which the database query is assigned;
obtaining, from the structured data dataset, values for the identified one or more attributes or metrics from the different data partitions in response to the plurality of database queries;
combining the obtained values from the different data partitions into a result dataset; and
applying the filter criteria that was generated using the natural language query to the result dataset to obtain a filtered result dataset; and
providing, for display on a user interface:
an indication of the identified one or more attributes or metrics from the structured data dataset; and
one or more graphical representations based on (i) the values in the filtered result dataset for the identified one or more attributes or metrics and (ii) the search results from the unstructured dataset.
11. The system of claim 10 , wherein the operations comprise:
generating one or more terms of the natural language query by tokenizing the natural language query; and
assigning a meaning or interpretation to each of the one or more terms,
wherein identifying the one or more attributes or metrics is based further on the meaning or interpretation assigned to each of the one or more terms.
12. The system of claim 10 , wherein the operations comprise:
identifying one or more keywords included in the one or more attributes or metrics or the natural language query; and
providing, for display on the user interface, graphical elements identifying the one or more keywords.
13. The system of claim 10 , wherein the operations comprise:
based on the one or more attributes or metrics, selecting a type of visualization for the obtained values for the one or more attributes or metrics,
wherein the one or more graphical representations include the selected type of visualization.
14. A non-transitory computer-readable recording medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
receiving a natural language query submitted by a user;
generating, using the natural language query, filter criteria to apply to structured data;
identifying search results from an unstructured dataset that are responsive to the natural language query;
identifying, based on the natural language query and the search results from the unstructured dataset, one or more attributes or metrics from a structured data dataset that correspond to the natural language query;
in response to receiving the natural language query and based on the identified one or more attributes or metrics, performing structured search operations including:
generating a plurality of database queries for processing on separate data partitions of the structured data dataset;
determining a partition attribute used to relate data among the data partitions;
using one or more values of the partition attribute to identify a particular data partition of the structured data dataset;
assigning each of the database queries to a different data partition of the structured data dataset including assigning a particular database query to the particular data partition identified using the one or more values of the partition attribute;
executing each of the database queries on the data partition to which the database query is assigned;
obtaining, from the structured data dataset, values for the identified one or more attributes or metrics from the different data partitions in response to the plurality of database queries;
combining the obtained values from the different data partitions into a result dataset; and
applying the filter criteria that was generated using the natural language query to the result dataset to obtain a filtered result dataset; and
providing, for display on a user interface:
an indication of the identified one or more attributes or metrics from the structured data dataset; and
one or more graphical representations based on (i) the values in the filtered result dataset for the identified one or more attributes or metrics and (ii) the search results from the unstructured dataset.
15. The non-transitory computer-readable recording medium of claim 14 , wherein the operations comprise:
generating one or more terms of the natural language query by tokenizing the natural language query; and
assigning a meaning or interpretation to each of the one or more terms,
wherein identifying the one or more attributes or metrics is based further on the meaning or interpretation assigned to each of the one or more terms.
16. The non-transitory computer-readable recording medium of claim 14 , wherein the operations comprise:
identifying one or more keywords included in the one or more attributes or metrics or the natural language query; and
providing, for display on the user interface, graphical elements identifying the one or more keywords.
17. The non-transitory computer-readable recording medium of claim 14 , wherein the operations comprise:
based on the one or more attributes or metrics, selecting a type of visualization for the obtained values for the one or more attributes or metrics,
wherein the one or more graphical representations include the selected type of visualization.