Question and answering on domain-specific tabular datasets

1 . A computer-implemented method for answering questions on domain-specific tabular datasets, comprising:

receiving a user question regarding a domain-specific tabular dataset;

generating a sequence of operations for answering the user question using a large language model, wherein each operation of the sequence of operations encodes a data operation, comprising:

selecting, from a set of parameterized basis functions, a final data operation, wherein the final data operation generates a response to the user question, and wherein each parameterized basis function of the set of parameterized basis functions comprises an operation and at least one parameter associated with data of the domain-specific tabular dataset; and

selecting, from the set of parameterized basis functions, one or more prior data operations, wherein the one or more prior data operations generate one or more prior outputs to be used by the final data operation to generate the response to the user question;

generating an output based on performing the sequence of operations on a tabular dataset, comprising:

for each data operation in the sequence of operations:

determining one or more parameters associated with the data operation;

referencing a metadata dictionary associated with the tabular dataset for each parameter of the one or more parameters associated with the data operation;

identifying a value from the tabular dataset for each parameter of the one or more parameters associated with the data operation based on the metadata dictionary; and

operating the data operation based on the identified value from the tabular dataset for each parameter of the one or more parameters; and

processing each data operation in the sequence of operations; and

determining a confidence score associated with the output; and

generating a natural language answer to the user question based on the output and the confidence score.

2 . The computer-implemented method of claim 1 , wherein determining the confidence score associated with the output comprises:

converting the sequence of operations to a reference question;

generating a first embedding vector representing the reference question;

generating a second embedding vector representing the user question; and

generating the confidence score based on a difference between the first embedding vector and the second embedding vector.

3 . The computer-implemented method of claim 2 , wherein the reference question is a question for which the sequence of operations answers.

4 . The computer-implemented method of claim 1 , wherein the sequence of operations comprises one or more of: a filtering operation; an aggregation operation; a trend operation; or an anomaly operation.

5 . A processing system, comprising: a memory comprising computer-executable instructions; and a processor configured to execute the computer-executable instructions and cause the processing system to:

receive a user question regarding a domain-specific tabular dataset;

generate a sequence of operations for answering the user question using a large language model, wherein each operation of the sequence of operations encodes a data operation, comprising:

selecting, from a set of parameterized basis functions, a final data operation, wherein the final data operation generates a response to the user question, and wherein each parameterized basis function of the set of parameterized basis functions comprises an operation and at least one parameter associated with data of the domain-specific tabular dataset; and

selecting, from the set of parameterized basis functions, one or more prior data operations, wherein the one or more prior data operations generate one or more prior outputs to be used by the final data operation to generate the response to the user question;

generate an output based on performing the sequence of operations on a tabular dataset, comprising:

for each data operation in the sequence of operations:

determining one or more parameters associated with the data operation;

referencing a metadata dictionary associated with the tabular dataset for each parameter of the one or more parameters associated with the data operation;

identifying a value from the tabular dataset for each parameter of the one or more parameters associated with the data operation based on the metadata dictionary; and

operating the data operation based on the identified value from the tabular dataset for each parameter of the one or more parameters; and

processing each data operation in the sequence of operations; and

determine a confidence score associated with the output; and

generate a natural language answer to the user question based on the output and the confidence score.

6 . The processing system of claim 5 , wherein in order to determine the confidence score associated with the output the processor is further configured to cause the processing system to:

convert the sequence of operations to a reference question;

generate a first embedding vector representing the reference question;

generate a second embedding vector representing the user question; and

generate the confidence score based on a difference between the first embedding vector and the second embedding vector.

7 . The processing system of claim 6 , wherein the reference question is a question for which the sequence of operations answers.

8 . The processing system of claim 5 , wherein the sequence of operations comprises one or more of: a filtering operation; an aggregation operation; a trend operation; or an anomaly operation.

9 . A computer-implemented method for answering questions on domain-specific tabular datasets, comprising:

receiving a user question regarding a domain-specific tabular dataset;

generating a sequence of operations for answering the user question using a large language model, wherein each operation of the sequence of operations encodes a data operation, comprising:

selecting, from a set of parameterized basis functions, a final data operation, wherein the final data operation generates a response to the user question, and wherein each parameterized basis function of the set of parameterized basis functions comprises an operation and at least one parameter associated with data of the domain-specific tabular dataset; and

selecting, from the set of parameterized basis functions, one or more prior data operations, wherein the one or more prior data operations generate one or more prior outputs to be used by the final data operation to generate the response to the user question;

generating an output based on performing the sequence of operations on a tabular dataset, comprising:

for each data operation in the sequence of operations:

determining one or more parameters associated with the data operation;

referencing a metadata dictionary associated with the tabular dataset for each parameter of the one or more parameters associated with the data operation;

identifying a value from the tabular dataset for each parameter of the one or more parameters associated with the data operation based on the metadata dictionary; and

operating the data operation based on the identified value from the tabular dataset for each parameter of the one or more parameters; and

processing each data operation in the sequence of operations;

determining a confidence score associated with the output, comprising:

converting the sequence of operations to a reference question;

generating a first embedding vector representing the reference question;

generating a second embedding vector representing the user question; and

generating the confidence score based on a difference between the first embedding vector and the second embedding vector; and

generating a natural language answer to the user question based on the output and the confidence score, wherein the confidence score is provided with the natural language answer.

10 . The computer-implemented method of claim 9 , wherein the reference question is a question for which the sequence of operations answers.

11 . The computer-implemented method of claim 9 , wherein the sequence of operations comprises one or more of: a filtering operation; an aggregation operation; a trend operation; or an anomaly operation.

12 . The computer-implemented method of claim 1 , further comprising:

translating, using the large language model, the user question regarding the domain- specific tabular dataset into a structured input comprising a sequence of tokens, wherein the structured input used by the large language model to generate the sequence of operations.

13 . The computer-implemented method of claim 1 , wherein generating the sequence of operations for answering the user question using the large language model further comprise determining an order for each operation of the sequence of operations.

14 . The computer-implemented method of claim 2 , wherein converting the sequence of operations to a reference question comprises:

identifying a summary associated with each operation in the sequence of operations; and

generating, using natural language processing, the reference question as a natural language reference question based on the summary associated with each operation in the sequence of operations.

15 . The computer-implemented method of claim 2 , wherein generating the confidence score based on the difference between the first embedding vector and the second embedding vector comprises determining a difference between the first embedding vector and the second embedding vector.

16 . The computer-implemented method of claim 9 , further comprising:

translating, using the large language model, the user question regarding the domain-specific tabular dataset into a structured input comprising a sequence of tokens, wherein the structured input used by the large language model to generate the sequence of operations.

17 . The computer-implemented method of claim 9 , wherein generating the sequence of operations for answering the user question using the large language model further comprise determining an order for each operation of the sequence of operations.

18 . The computer-implemented method of claim 10 , wherein converting the sequence of operations to a reference question comprises:

identifying a summary associated with each operation in the sequence of operations; and

generating, using natural language processing, the reference question as a natural language reference question based on the summary associated with each operation in the sequence of operations.