MERGE-SORT LLM PROMPTING FOR DATA ANALYTICS
An online system enables large language models (LLMs) to perform analytics on large datasets that exceed the LLM's context window by employing a recursive merge-sort prompting approach. Upon receiving a free-text analysis request from a user, the system generates an initialization prompt for the LLM to create a merge prompt template. The dataset is recursively divided into portions, which may be equal or semantically segmented subsets, and each portion is analyzed by the LLM. If a portion still exceeds the context window, the recursive process continues on its subsets. Outputs from each analysis are merged using a merge prompt template, which may include fields for outputs, data, and descriptions of subsets. The merge prompt template is used to generate a merge prompt, which is provided to the LLM to synthesize a final response.
1 . A method comprising:
receiving, at an online system, an analysis request from a user, wherein the analysis request comprises free text requesting analytics of a set of data stored by the online system in a database;
generating an initialization prompt based on the analysis request, wherein the initialization prompt comprises text instructing a large language model to generate a merge prompt template based on the analysis request;
transmitting the initialization prompt to the large language model;
receiving a response to the initialization prompt, wherein the response comprises the merge prompt template;
generating a response to the analysis request by applying a recursive process to the set of data, wherein the recursive process comprises:
determining whether a terminating condition for the recursive process is met;
and
responsive to determining that the terminating condition is not met:
applying the recursive process to a first portion of the set of data to generate a first output, wherein applying the recursive process to the first portion comprises applying the analysis request to the first portion of the set of data using the large language model;
applying the recursive process to a second portion of the set of data to generate a second output, wherein applying the recursive process to the second portion comprises applying the analysis request to the second portion of the set of data using the large language model;
generating a merge prompt based on the merge prompt template, the first output, and the second output;
transmitting the merge prompt to the large language model; and
receiving a response to the merge prompt from the large language model; and
transmitting a response to the analysis request based on the received response to the merge prompt.
2 . The method of claim 1 , wherein the first portion and the second portion each comprise substantially equal subsets of the set of data.
3 . The method of claim 1 , wherein the first portion and the second portion each comprise a semantically segmented subset of the set of data.
4 . The method of claim 1 , wherein receiving the analysis request from the user comprises:
receiving the analysis request through a chat user interface presented on a display of a client device of the user.
5 . The method of claim 1 , further comprising:
receiving, through a user interface presented to the user on a display of a client device, input parameters for the recursive prompting; and
generating the initialization prompt based on the input parameters.
6 . The method of claim 1 , wherein the merge prompt template includes fields for outputs generated by applying the recursive process to the first portion and the second portion.
7 . The method of claim 6 , wherein the merge prompt template includes fields for text descriptions of the first portion and the second portion in association with the corresponding fields for the outputs.
8 . The method of claim 1 , wherein the merge prompt template includes a field for some or all of the set of data.
9 . The method of claim 1 , wherein applying the recursive process to the first portion comprises:
determining whether the first portion meets the terminating condition.
10 . The method of claim 9 , further comprising:
responsive to determining that the first portion does not meet the terminating condition, applying the recursive process to subsets of the first portion.
11 . A non-transitory computer-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising:
receiving, at an online system, an analysis request from a user, wherein the analysis request comprises free text requesting analytics of a set of data stored by the online system in a database;
generating an initialization prompt based on the analysis request, wherein the initialization prompt comprises text instructing a large language model to generate a merge prompt template based on the analysis request;
transmitting the initialization prompt to the large language model;
receiving a response to the initialization prompt, wherein the response comprises the merge prompt template;
generating a response to the analysis request by applying a recursive process to the set of data, wherein the recursive process comprises:
determining whether a terminating condition for the recursive process is met; and
responsive to determining that the terminating condition is not met:
applying the recursive process to a first portion of the set of data to generate a first output, wherein applying the recursive process to the first portion comprises applying the analysis request to the first portion of the set of data using the large language model;
applying the recursive process to a second portion of the set of data to generate a second output, wherein applying the recursive process to the second portion comprises applying the analysis request to the second portion of the set of data using the large language model;
generating a merge prompt based on the merge prompt template, the first output, and the second output;
transmitting the merge prompt to the large language model; and
receiving a response to the merge prompt from the large language model; and
transmitting a response to the analysis request based on the received response to the merge prompt.
12 . The computer-readable medium of claim 11 , wherein the first portion and the second portion each comprise substantially equal subsets of the set of data.
13 . The computer-readable medium of claim 11 , wherein the first portion and the second portion each comprise a semantically segmented subset of the set of data.
14 . The computer-readable medium of claim 11 , wherein receiving the analysis request from the user comprises:
receiving the analysis request through a chat user interface presented on a display of a client device of the user.
15 . The computer-readable medium of claim 11 , further comprising:
receiving, through a user interface presented to the user on a display of a client device, input parameters for the recursive prompting; and
generating the initialization prompt based on the input parameters.
16 . The computer-readable medium of claim 11 , wherein the merge prompt template includes fields for outputs generated by applying the recursive process to the first portion and the second portion.
17 . The computer-readable medium of claim 16 , wherein the merge prompt template includes fields for text descriptions of the first portion and the second portion in association with the corresponding fields for the outputs.
18 . The computer-readable medium of claim 11 , wherein the merge prompt template includes a field for some or all of the set of data.
19 . The computer-readable medium of claim 11 , wherein applying the recursive process to the first portion comprises:
determining whether the first portion meets the terminating condition.
20 . The computer-readable medium of claim 19 , further comprising:
responsive to determining that the first portion does not meet the terminating condition, applying the recursive process to subsets of the first portion.