Natural language interface databases
Embodiments are disclosed for correcting a natural language interface database (NLIDB) system. The techniques include receiving feedback indicating that an answer provided in response to a question for an NLIDB system is inaccurate. The techniques further include finding an ontology element for a datastore of the NLIDB system that matches to the feedback. The techniques also include selecting candidate annotations for the NLIDB system based on the ontology element and a data type of the ontology element. Additionally, the techniques include generating a question-answer (QA) pair for each of the candidate annotations. Further, the techniques include adding one of the candidate annotations to annotations for a natural language query (NLQ) engine of the NLIDB system based on a client verification of the QA pair.
1. A computer-implemented method comprising:
receiving feedback indicating an old answer provided in response to a question for a natural language interface database (NLIDB) system is inaccurate, wherein the NLIDB:
generates the old answer using an old structured query language (SQL) query; and
generates the old SQL query based on an old question;
finding an ontology element for a datastore of the NLIDB system that matches to the feedback indicating the old answer is inaccurate;
selecting a plurality of candidate annotations for the NLIDB system based on the ontology element and a data type of the ontology element;
generating a question-answer (QA) pair for each of the candidate annotations;
adding one of the candidate annotations to a plurality of annotations for a natural language query (NLQ) engine of the NLIDB system based on a client verification of the QA pair;
generating a new structured query language (SQL) query for the question based on one of the QA-pairs, and the one candidate annotation; and
generating a new answer for the question that is more accurate than the old answer by executing the new SQL query.
2. The method of claim 1 , wherein the ontology element comprises an alias that maps a word of the question to a table of the datastore.
3. The method of claim 1 , wherein the feedback indicating the old answer is inaccurate indicates a portion of the question.
4. The method of claim 1 , wherein the data type is numeric, and wherein the candidate annotations comprise SELECT, SUM, MAX, MIN, AVERAGE, and ORDERBY clauses.
5. The method of claim 1 , wherein the data type is not numeric, and wherein the feedback indicating the old answer is inaccurate is within a short edit distance of a data instance value in the datastore, and wherein the candidate annotation comprises a WHERE clause.
6. The method of claim 1 , wherein the data type is not numeric, and wherein the feedback indicating the old answer is inaccurate is not within a short edit distance of a data instance value in the datastore, and wherein the candidate annotations comprise SELECT and GROUPBY clauses.
7. The method of claim 1 , wherein the NLQ engine is a machine-learning system.
8. The method of claim 1 , wherein the NLQ engine is a rule-based system.
9. A computer program product comprising program instructions stored on a computer readable storage medium, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:
receiving feedback indicating an old answer provided in response to a question for a natural language interface database (NLIDB) system is inaccurate, wherein the NLIDB:
generates the old answer using an old structured query language (SQL) query; and
generates the old SQL query based on an old question;
finding an ontology element for a datastore of the NLIDB system that matches to the received feedback indicating the old answer is inaccurate;
selecting a plurality of candidate annotations for the NLIDB system based on the ontology element and a data type of the ontology element;
generating a question-answer (QA) pair for each of the candidate annotations;
adding one of the candidate annotations to a plurality of annotations for a natural language query (NLQ) engine of the NLIDB system based on a client verification of the QA pair;
generating a new structured query language (SQL) query for the question based on one of the QA-pairs, and the one candidate annotation; and
generating a new answer for the question that is more accurate than the old answer by executing the new SQL query.
10. The computer program product of claim 9 , wherein the ontology element comprises an alias that maps a word of the question to a table of the datastore.
11. The computer program product of claim 9 , wherein the feedback indicating the old answer is inaccurate further indicates a portion of the question.
12. The computer program product of claim 9 , wherein the data type is numeric, and wherein the candidate annotations comprise SELECT, SUM, MAX, MIN, AVERAGE, and ORDERBY clauses.
13. The computer program product of claim 9 , wherein the data type is not numeric, and wherein the feedback indicating the old answer is inaccurate is within a short edit distance of a data instance value in the datastore, and wherein the candidate annotation comprises a WHERE clause.
14. The computer program product of claim 9 , wherein the data type is not numeric, and wherein the feedback indicating the old answer is inaccurate is not within a short edit distance of a data instance value in the datastore, and wherein the candidate annotations comprise SELECT and GROUPBY clauses.
15. A system comprising:
a computer processing circuit; and
a computer-readable storage medium storing instructions, which, when executed by the computer processing circuit, are configured to cause the computer processing circuit to perform a method comprising:
receiving feedback indicating an old answer provided in response to a question for a natural language interface database (NLIDB) system is inaccurate, wherein the NLIDB:
generates the old answer using an old structured query language (SQL) query; and
generates the old SQL query based on an old question;
finding an ontology element for a datastore of the NLIDB system that matches to the feedback indicating the old answer is inaccurate;
selecting a plurality of candidate annotations for the NLIDB system based on the ontology element and a data type of the ontology element;
generating a question-answer (QA) pair for each of the candidate annotations;
adding one of the candidate annotations to a plurality of annotations for a natural language query (NLQ) engine of the NLIDB system based on a client verification of the QA pair;
generating a new structured query language (SQL) query for the question based on one of the QA-pairs, and the one candidate annotation; and
generating a new answer for the question that is more accurate than the old answer by executing the new SQL query.
16. The system of claim 15 , wherein the ontology element comprises an alias that maps a word of the question to a table of the datastore.
17. The system of claim 15 , wherein the feedback indicating the old answer is inaccurate further indicates a portion of the question.
18. The system of claim 15 , wherein the data type is numeric, and wherein the candidate annotations comprise SELECT, SUM, MAX, MIN, AVERAGE, and ORDERBY clauses.
19. The system of claim 15 , wherein the data type is not numeric, and wherein the feedback indicating the old answer is inaccurate is within a short edit distance of a data instance value in the datastore, and wherein the candidate annotation comprises a WHERE clause.
20. The system of claim 15 , wherein the data type is not numeric, and wherein the feedback indicating the old answer is inaccurate is not within a short edit distance of a data instance value in the datastore, and wherein the candidate annotations comprise SELECT and GROUPBY clauses.