Techniques for data-driven computation to enable the automatic generation of data-driven insights in response to natural language queries
Certain aspects of the disclosure provide techniques for code generation and execution for data-driven computation. A method generally includes obtaining: structured data associated with a query; and a plan that defines an analysis sequence for generation of data-driven output that is responsive to the query, wherein the analysis sequence comprises a plurality of steps based on the structured data; generating, based on the plan, prompt(s), each respective prompt comprising at least: instructions for generating a set of executable code configured to perform computation(s) in accordance with one or more of the steps; and a data profile for the structured data; prompting a language model (LM), with each respective prompt, to generate each set of executable code based on at least the data profile; executing each set of executable code to produce computed result(s); and generating data-driven output based on at least one computed result.
1 . A method of data-driven code generation and execution, comprising:
obtaining:
structured data associated with a natural language query; and
a plan that defines an analysis sequence for generation of data-driven output that is responsive to the natural language query, wherein the analysis sequence comprises a plurality of steps based on the structured data;
generating, based on the plan, one or more prompts, each respective prompt of the one or more prompts comprising at least:
instructions for generating a set of executable code configured to perform at least one computation in accordance with one or more steps of the plurality of steps; and
a data profile for the structured data;
prompting a language model (LM), with each respective prompt, to generate the set of executable code, associated with each respective prompt, based on at least the data profile;
executing the set of executable code associated with each respective prompt to produce one or more computed results; and
generating the data-driven output based on at least one computed result of the one or more computed results.
2 . The method of claim 1 , wherein:
the data profile included in each respective prompt comprises:
a subset of the structured data; or
a summarization of the structured data; and
a size of the data profile included in each respective prompt is based on a context window of the language model.
3 . The method of claim 1 , further comprising:
generating, by an embedding model, a query embedding for the natural language query;
generating, by the embedding model, a plurality of portion embeddings for a plurality of portions of the structured data;
determining a similarity score for each respective portion embedding of the plurality of portion embeddings based on a similarity between the query embedding and each respective portion embedding;
identifying the similarity score determined for a first subset of the plurality of portion embeddings, associated with a first subset of the plurality of portions of the structured data, satisfies a similarity threshold; and
generating the data profile for each respective prompt based on the first subset of the plurality of portions of the structured data.
4 . The method of claim 1 , wherein executing the set of executable code, associated with each respective prompt, comprises executing, by an analytical engine, the set of executable code, associated with each respective prompt.
5 . The method of claim 1 , wherein prompting the LM, with each respective prompt, to generate the set of executable code, and executing the set of executable code associated with each respective prompt comprises:
for a first prompt of the one or more prompts:
executing the set of executable code associated with the first prompt to produce a first computed result of the one or more computed results; and
updating the data profile to include the first computed result;
for each respective remaining prompt of the one or more prompts and in accordance with an execution order associated with the analysis sequence of the plan:
executing the set of executable code associated with the respective remaining prompt to produce a second computed result of the one or more computed results; and
updating the data profile to include the second computed result.
6 . The method of claim 5 , wherein the first prompt comprises the instructions for generating the set of executable code configured to perform the at least one computation in accordance with a first-in-time computational step of the plurality of steps.
7 . The method of claim 1 , wherein the set of executable code associated with each respective prompt comprises at least one of:
structured query language (SQL) code; or
Python code.
8 . The method of claim 1 , further comprising displaying, via a user interface of a computing device, the data-driven output.
9 . The method of claim 8 , wherein generating the data-driven output comprises generating a narrative explanation that references the at least one computed result.
10 . The method of claim 8 , wherein generating the data-driven output comprises generating a visualization based on the at least one computed result.
11 . The method of claim 1 , wherein:
the natural language query relates to a business insight comprising at least one of a financial measure, a performance assessment, or a decision support output; and
the at least one computed result comprises information associated with the business insight.
12 . A processing system, comprising: memory comprising computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to:
obtain:
structured data associated with a natural language query; and
a plan that defines an analysis sequence for generation of data-driven output that is responsive to the natural language query, wherein the analysis sequence comprises a plurality of steps based on the structured data;
generate, based on the plan, one or more prompts, each respective prompt of the one or more prompts comprising at least:
instructions for generating a set of executable code configured to perform at least one computation in accordance with one or more steps of the plurality of steps; and
a data profile for the structured data;
prompt a language model (LM), with each respective prompt, to generate the set of executable code, associated with each respective prompt, based on at least the data profile;
execute the set of executable code associated with each respective prompt to produce one or more computed results; and
generate data-driven output based on at least one computed result of the one or more computed results.
13 . The processing system of claim 12 , wherein:
the data profile included in each respective prompt comprises:
a subset of the structured data; or
a summarization of the structured data; and
a size of the data profile included in each respective prompt is based on a context window of the language model.
14 . The processing system of claim 12 , wherein the one or more processors are configured to execute the computer-executable instructions and cause the processing system to:
generate, by an embedding model, a query embedding for the natural language query;
generate, by the embedding model, a plurality of portion embeddings for a plurality of portions of the structured data;
determine a similarity score for each respective portion embedding of the plurality of portion embeddings based on a similarity between the query embedding and each respective portion embedding;
identify the similarity score determined for a first subset of the plurality of portion embeddings, associated with a first subset of the plurality of portions of the structured data, satisfies a similarity threshold; and
generate the data profile for each respective prompt based on the first subset of the plurality of portions of the structured data.
15 . The processing system of claim 12 , wherein to cause the processing system to execute the set of executable code, associated with each respective prompt, the one or more processors are configured to execute the computer-executable instructions and cause the processing system to execute, by an analytical engine, the set of executable code, associated with each respective prompt.
16 . The processing system of claim 12 , wherein to cause the processing system to prompt the LM, with each respective prompt, to generate the set of executable code, and to cause the processing system to execute the set of executable code associated with each respective prompt, the one or more processors are configured to execute the computer-executable instructions and cause the processing system to:
for a first prompt of the one or more prompts:
execute the set of executable code associated with the first prompt to produce a first computed result of the one or more computed results; and
update the data profile to include the first computed result;
for each respective remaining prompt of the one or more prompts and in accordance with an execution order associated with the analysis sequence of the plan:
execute the set of executable code associated with the respective remaining prompt to produce a second computed result of the one or more computed results; and
update the data profile to include the second computed result.
17 . The processing system of claim 16 , wherein the first prompt comprises the instructions for generating the set of executable code configured to perform the at least one computation in accordance with a first-in-time computational step of the plurality of steps.
18 . The processing system of claim 12 , wherein the set of executable code associated with each respective prompt comprises at least one of:
structured query language (SQL) code; or
Python code.
19 . The processing system of claim 12 , wherein the one or more processors are configured to execute the computer-executable instructions and cause the processing system to display, via a user interface of a computing device, the data-driven output.
20 . The processing system of claim 19 , wherein to cause the processing system to generate the data-driven output, the one or more processors are configured to execute the computer-executable instructions and cause the processing system to generate a narrative explanation that references the at least one computed result.