Trained sequence-to-sequence conversion of database queries
Methods and systems are provided for sequence-to-sequence conversion from unstructured search queries to structured database queries, so that lay persons may retrieve information from relational databases without specialized knowledge of database query languages. An encoder module and a decoder module of a learning model are trained to convert an unstructured search query to an intermediate feature vector by computing co-attention and self-attention based on a context string and a database schema, encoding the database schema in the context string by application of self-attention between the context string containing tokens of the database schema with learned structural attention heads which relate the token to logic of the database. Training is performed using labeled training datasets which include structured database queries which are normalized by parsing into a semantic representation thereof, followed by linearization.
1. A method comprising:
computing co-attention for an unstructured search query and a context string to learn a correspondence between a token of the unstructured search query and a token of the context string;
computing self-attention for the unstructured search query to learn a correspondence between a plurality of tokens of the unstructured search query;
computing self-attention for the context string based on trained structural attention heads comprising at least a structural attention head trained across all tokens of the context string and a plurality of structural attention heads trained for each respective individual token of the context string, a structural attention head comprising a weight vector, wherein each feature of the weight vector corresponds to a different token of the context string;
computing the self-attention-computed search query and the self-attention-computed context string by a multi-pointer-generator; and
outputting a structured database query based on minimizing output of a cost function, causing a database management system to return results of the structured database query answering the unstructured search query.
2. The method of claim 1 , wherein the trained structural attention heads are learned using a training dataset comprising normalized labeled structural database queries.
3. The method of claim 2 , wherein the labeled structural database queries are normalized by parsing into semantic representations thereof.
4. The method of claim 3 , wherein the labeled structural database queries are further normalized by linearization into strings.
5. The method of claim 1 , wherein the trained structural attention heads comprise a plurality of structural attention heads trained across each token of a tokenized context string.
6. The method of claim 1 , wherein the trained structural attention heads comprise a plurality of structural attention heads each trained for an individual token of the tokenized context string.
7. The method of claim 6 , wherein each of the plurality of structural attention heads comprises a weight set indicating a data type of a field corresponding to the respective individual token or a weight set indicating that the respective individual token is a foreign key.
8. A system comprising:
one or more processors; and
memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors that, when executed by the one or more processors, perform associated operations, the computer-executable modules comprising:
an encoding module further comprising:
a co-attention submodule configured to compute co-attention for an unstructured search query and a context string to learn a correspondence between a token of the unstructured search query and a token of the context string;
a first self-attention submodule configured to compute self-attention for the unstructured search query to learn a correspondence between a plurality of tokens of the unstructured search query; and
a second self-attention submodule configured to compute self-attention for the context string based on trained structural attention heads comprising at least a structural attention head trained across all tokens of the context string and a plurality of structural attention heads trained for each respective individual token of the context string, a structural attention head comprising a weight vector, wherein each feature of the weight vector corresponds to a different token of the context string; and
a decoder module configured to compute the self-attention-computed search query and the self-attention-computed context string by a multi-pointer-generator and output a structured database query based on minimizing output of a cost function, causing a database management system to return results of the structured database query answering the unstructured search query.
9. The system of claim 8 , the trained structural attention heads are learned using a training dataset comprising normalized labeled structural database queries.
10. The method of claim 9 , wherein the labeled structural database queries are normalized by parsing into semantic representations thereof.
11. The method of claim 10 , wherein the labeled structural database queries are further normalized by linearization into strings.
12. The method of claim 8 , wherein the trained structural attention heads comprise a plurality of structural attention heads trained across each token of a tokenized context string.
13. The method of claim 8 , wherein the trained structural attention heads comprise a plurality of structural attention heads each trained for an individual token of the tokenized context string.
14. The method of claim 13 , wherein each of the plurality of structural attention heads comprises a weight set indicating a data type of a field corresponding to the respective individual token or a weight set indicating that the respective individual token is a foreign key.
15. A computer-readable storage medium storing computer-readable instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising:
computing co-attention for an unstructured search query and a context string to learn a correspondence between a token of the unstructured search query and a token of the context string;
computing self-attention for the unstructured search query to learn a correspondence between a plurality of tokens of the unstructured search query;
computing self-attention for the context string based on trained structural attention heads comprising at least a structural attention head trained across all tokens of the context string and a plurality of structural attention heads trained for each respective individual token of the context string, a structural attention head comprising a weight vector, wherein each feature of the weight vector corresponds to a different token of the context string;
computing the self-attention-computed search query and the self-attention-computed context string by a multi-pointer-generator; and
outputting a structured database query based on minimizing output of a cost function, causing a database management system to return results of the structured database query answering the unstructured search query.
16. The computer-readable storage medium of claim 15 , wherein the labeled structural database queries are normalized by parsing into semantic representations thereof.
17. The computer-readable storage medium of claim 16 , wherein the labeled structural database queries are normalized by parsing into semantic representations thereof.
18. The computer-readable storage medium of claim 17 , wherein the labeled structural database queries are further normalized by linearization into strings.
19. The computer-readable storage medium of claim 15 , wherein the trained structural attention heads comprise a plurality of structural attention heads trained across each token of a tokenized context string.
20. The computer-readable storage medium of claim 15 , wherein the trained structural attention heads comprise a plurality of structural attention heads each trained for an individual token of the tokenized context string.