System, method, and computer program for augmenting multi-turn text-to-SQL datasets with self-play

1. A method for augmenting multi-turn text-to-SQL datasets, performed by at least one processor and comprising:

generating an SQL-to-text model to converse with a text-to-SQL model;

pre-training the SQL-to-text model and the text-to-SQL model based on input training data;

sampling an SQL query as a goal query of an interaction between the SQL-to-text model and the text-to-SQL model;

generating the interaction based on the goal query, a current utterance, previous utterances in the interaction, an SQL query from a preceding turn of the interaction, and a serialized database; and

filtering the interaction based on a last turn of the interaction and the goal query, wherein the interaction is kept or filtered out based on a similarity score between the last turn of the interaction and the goal query,

wherein generating the interaction comprises concatenating the goal query, the SQL query from the preceding turn of the interaction, the previous utterances in the interaction, and a schema of the serialized database, and using a result of the concatenating as an input to the SQL-to-text model for generating a next utterance in the interaction using the SQL-to-text model.

2. The method of claim 1 , wherein sampling the SQL query as the goal query comprises:

building SQL templates using SQL queries in the input training data by replacing column and value mentions in the SQL queries with typed slots; and

sampling an unseen database and fill the typed slots with columns and values from the sampled unseen database to form the goal query.

3. The method of claim 1 , further comprising:

concatenating the goal query with an empty context and a schema from the serialized database to generate a first value; and

inputting the first value into the SQL-to-text model to produce a first user utterance of the interaction.

4. The method of claim 1 , further comprising:

padding an utterance from the last turn of the interaction with a stop interaction symbol, wherein the interaction ends when the SQL-to-text model decodes the stop interaction symbol.

5. The method of claim 1 , wherein:

when the similarity score is above a threshold value, the interaction is determined to be grounded to the goal query and the interaction is kept, and

when the similarity score is below the threshold value, the interaction is determined not to be grounded to the goal query and the interaction is filtered out.

6. The method of claim 1 , further comprising re-training the SQL-to-text model and the text-to-SQL model based on the input training data and the filtered interaction.

7. An apparatus for augmenting multi-turn text-to-SQL datasets, the apparatus comprising:

at least one memory configured to store computer program code; and

at least one processor configured to read the computer program code and operate as instructed by the computer program code, the computer program code including:

generating code configured to cause the at least one processor to generate an SQL-to-text model to converse with a text-to-SQL model;

training code configured to cause the at least one processor to pre-train the SQL-to-text model and the text-to-SQL model based on input training data;

first sampling code configured to cause the at least one processor to sample an SQL query as a goal query of an interaction between the SQL-to-text model and the text-to-SQL model;

synthetic interaction generating code configured to cause the at least one processor to generate the interaction based on the goal query, a current utterance, previous utterances in the interaction, an SQL query from a preceding turn of the interaction, and a serialized database;

filtering code configured to cause the at least one processor to filter the interaction based on a last turn of the interaction and the goal query, wherein the interaction is kept or filtered out based on a similarity score between the last turn of the interaction and the goal query; and

utterance generating code configured to cause the at least one processor to concatenate the goal query, the SQL query from the preceding turn of the interaction, the previous utterances in the interaction, and a schema of the serialized database, and use a result of the concatenation to generate a next utterance in the interaction using the SQL-to-text model.

8. The apparatus of claim 7 , the computer program code further including:

building code configured to cause the at least one processor to build SQL templates using SQL queries in the input training data by replacing column and value mentions in the SQL queries with typed slots; and

second sampling code configured to cause the at least one processor to sample an unseen database and fill the typed slots with columns and values from the sampled unseen database to form the goal query.

9. The apparatus of claim 7 , the computer program code further including:

concatenating code configured to cause the at least one processor to concatenate the goal query with an empty context and a schema from the serialized database to generate a first value; and

first user utterance code configured to cause the at least one processor to input the first value into the SQL-to-text model to produce a first user utterance of the interaction.

10. The apparatus of claim 7 , the computer program code further including:

stopping code configured to cause the at least one processor to pad an utterance from the last turn of the interaction with a stop interaction symbol, wherein the interaction ends when the SQL-to-text model decodes the stop interaction symbol.

11. The apparatus of claim 7 , wherein:

when the similarity score is above a threshold value, the interaction is determined to be grounded to the goal query and the interaction is kept, and

when the similarity score is below the threshold value, the interaction is determined not to be grounded to the goal query and the interaction is filtered out.

12. The apparatus of claim 7 , the computer program code further including:

re-training code configured to cause the at least one processor to re-train the SQL-to-text model and the text-to-SQL model based on the input training data and the filtered interaction.

13. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor of an apparatus for augmenting multi-turn text-to-SQL datasets, cause the at least one processor to:

generate an SQL-to-text model to converse with a text-to-SQL model;

pre-train the SQL-to-text model and the text-to-SQL model based on input training data;

sample an SQL query as a goal query of an interaction between the SQL-to-text model and the text-to-SQL model;

generate the interaction based on the goal query, a current utterance, previous utterances in the interaction, an SQL query from a preceding turn of the interaction, and a serialized database;

filter the interaction based on a last turn of the interaction and the goal query, wherein the interaction is kept or filtered out based on a similarity score between the last turn of the interaction and the goal query,

wherein generating the interaction comprises concatenating the goal query, the SQL query from the preceding turn of the interaction, the previous utterances in the interaction, and a schema of the serialized database, and using a result of the concatenating as an input to the SQL-to-text model for generating a next utterance in the interaction using the SQL-to-text model.

14. The non-transitory computer-readable medium of claim 13 , wherein the instructions further cause the at least one processor to:

build SQL templates using SQL queries in the input training data by replacing column and value mentions in the SQL queries with typed slots; and

sample an unseen database and fill the typed slots with columns and values from the sampled unseen database to form the goal query.

15. The non-transitory computer-readable medium of claim 13 , wherein the instructions further cause the at least one processor to:

concatenate the goal query with an empty context and a schema from the serialized database to generate a first value;

input the first value into the SQL-to-text model to produce a first user utterance of the interaction; and

pad an utterance from the last turn of the interaction with a stop interaction symbol, wherein the interaction ends when the SQL-to-text model decodes the stop interaction symbol.

16. The non-transitory computer-readable medium of claim 13 , wherein:

when the similarity score is above a threshold value, the interaction is determined to be grounded to the goal query and the interaction is kept, and

when the similarity score is below the threshold value, the interaction is determined not to be grounded to the goal query and the interaction is filtered out.

17. The non-transitory computer-readable medium of claim 14 , wherein the instructions further cause the at least one processor to:

re-train the SQL-to-text model and the text-to-SQL model based on the input training data and the filtered interaction.