Multi-stage processing for large language model to answer math questions more accurately

1 . A method of answering a natural language prompt using a large language model, comprising:

receiving, at an interface between a user and a large language model, an original natural language prompt including a plurality of facts from the user;

generating, at the interface, a series of contextual sub-questions based on the original natural language prompt using the large language model, wherein the generating comprises:

querying the large language model with a first contextual sub-question about the natural language prompt; and

querying the large language model with a second contextual sub-question that includes an outcome of the first sub-question;

providing the contextual sub-questions to the large language model to obtain contextual answers;

applying the contextual sub-questions against the original natural language prompt with the contextual answers as a refined natural language prompt to the large language model in a reverse order of the series, wherein applying the contextual sub-questions against the original natural language prompt with the contextual answers as a refined natural language prompt to the large language model in the reverse order of the series comprises:

adding the contextual answer of a last sub-question to the original natural language prompt to form a first level refined natural language prompt;

querying the large language model with the first level refined natural language prompt;

saving an answer from the large language model to the first level refined natural language prompt as a saved context;

combining the saved context with the contextual answer from another sub-question to generate one or more second level refined natural language prompts;

querying the large language model with the second level refined natural language prompt; and

updating the saved context with a result of the second level refined natural language prompt; and

outputting, to the user, a final answer from the large language model to a terminal state of the refined natural language prompt.

2 . The method of claim 1 , wherein the final answer corresponds to a recommendation.

3 . The method of claim 1 , further comprising determining based on semantic assessment using a taxonomical dictionary of the large language model that the original natural language prompt includes a question related to arithmetic reasoning.

4 . The method of claim 3 , wherein the original natural language prompt includes at least numerical values and an operation or comparison according to the taxonomical dictionary.

5 . The method of claim 1 , wherein generating the series of contextual sub-questions based on the original natural language prompt using the large language model comprises:

querying the large language model with a first sub-question about a goal of the original natural language prompt;

querying the large language model with a second sub-question, using an outcome of the first sub-question, about an action of the original natural language prompt;

querying the large language model with a third sub-question, using an outcome of the second sub-question, about a subject of the action; and

querying the large language model with a fourth sub-question, using the outcome of the second sub-question, about an object of the action.

6 . The method of claim 1 , wherein generating the series of contextual sub-questions based on the original natural language prompt using the large language model comprises:

querying the large language model with a first sub-question to identify one or more verbs of the original natural language prompt; and

querying the large language model with one or more additional sub-questions, using an outcome of a previous sub-question, to identify nouns related to the one or more verbs until all of the one or more verbs have been queried.

7 . The method of claim 1 , wherein applying the contextual sub-questions against the original natural language prompt with the contextual answers as a refined natural language prompt to the large language model in the reverse order of the series comprises:

combining the saved context with the original natural language prompt to form the terminal state of the refined natural language prompt.

8 . An apparatus comprising:

a memory; and

a processor coupled with the memory and configured to:

receive, at an interface between a user and a large language model, an original natural language prompt including a plurality of facts from the user;

generate a series of contextual sub-questions based on the original natural language prompt using the large language model, wherein to generate the series of contextual sub-questions based on the original natural language prompt using the large language model, the processor is further configured to:

query the large language model with a first contextual sub-question about the natural language prompt; and

query the large language model with a second contextual sub-question that includes an outcome of the first sub-question;

provide the contextual sub-questions to the large language model to obtain contextual answers;

apply the contextual sub-questions against the original natural language prompt with the contextual answers as a refined natural language prompt to the large language model in a reverse order of the series, wherein to apply the contextual sub-questions against the original natural language prompt with the contextual answers as a refined natural language prompt to the large language model in the reverse order of the series, the processor is further configured to:

add the contextual answer of a last sub-question to the original natural language prompt to form a first level refined natural language prompt;

query the large language model with the first level refined natural language prompt;

save an answer from the large language model to the first level refined natural language prompt as a saved context;

combine the saved context with the contextual answer from another sub-question to generate one or more second level refined natural language queries;

query the large language model with the second level refined natural language prompt; and

update the saved context with a result of the second level refined natural language prompt; and

output, to the user, a final answer from the large language model to a terminal state of the refined natural language prompt.

9 . The apparatus of claim 8 , wherein the processor is further configured to determine based on semantic assessment using a taxonomical dictionary of the large language model that the original natural language prompt includes a question related to arithmetic reasoning.

10 . The apparatus of claim 9 , wherein the original natural language prompt includes at least numerical values and an operation or comparison according to the taxonomical dictionary.

11 . The apparatus of claim 8 , wherein to generate the series of contextual sub-questions based on the original natural language prompt using the large language model, the processor is further configured to:

query the large language model with a first sub-question about a goal of the original natural language prompt;

query the large language model with a second sub-question, using an outcome of the first sub-question, about an action of the original natural language prompt;

query the large language model with a third sub-question, using an outcome of the second sub-question, about a subject of the action; and

query the large language model with a fourth sub-question, using the outcome of the second sub-question, about an object of the action.

12 . The apparatus of claim 8 , wherein to generate the series of contextual sub-questions based on the original natural language prompt using the large language model, the processor is further configured to:

query the large language model with a first sub-question to identify one or more verbs of the original natural language prompt; and

query the large language model with one or more additional sub-questions, using an outcome of a previous sub-question, to identify nouns related to the one or more verbs until all of the one or more verbs have been queried.

13 . The apparatus of claim 8 , wherein to apply the contextual sub-questions against the original natural language prompt with the contextual answers as a refined natural language prompt to the large language model in the reverse order of the series, the processor is further configured to:

combine the saved context with the original natural language prompt to form the terminal state of the refined natural language prompt.

14 . A non-transitory computer-readable medium having computer-executable instructions stored thereon that when executed by a computer processor cause the computer processor to:

receive, at an interface between a user and a large language model, an original natural language prompt including a plurality of facts from the user;

generate a series of contextual sub-questions based on the original natural language prompt using the large language model, wherein the instructions to generate the series of contextual sub-questions comprise instructions to:

query the large language model with a first contextual sub-question about the natural language prompt; and

query the large language model with a second contextual sub-question that includes an outcome of the first sub-question;

provide the contextual sub-questions to the large language model to obtain contextual answers;

apply the contextual sub-questions against the original natural language prompt with the contextual answers as a refined natural language prompt to the large language model in a reverse order of the series, wherein the instructions to apply the contextual sub-questions against the original natural language prompt with the contextual answers as a refined natural language prompt to the large language model in the reverse order of the series comprise instructions to:

add the contextual answer of a last sub-question to the original natural language prompt to form a first level refined natural language prompt;

query the large language model with the first level refined natural language prompt;

save an answer from the large language model to the first level refined natural language prompt as a saved context;

combine the saved context with the contextual answer from another sub-question to generate one or more second level refined natural language queries;

query the large language model with the second level refined natural language prompt; and

update the saved context with a result of the second level refined natural language prompt; and

output, to the user, a final answer from the large language model to a terminal state of the refined natural language prompt.

15 . The non-transitory computer-readable medium of claim 14 , further comprising instructions to determine based on semantic assessment using a taxonomical dictionary of the large language model that the original natural language prompt includes a question related to arithmetic reason.