Adaptive confidence-based reasoning control for large language models

1 . A method comprising:

transmitting a prompt to a large language model (LLM), the prompt comprising an instruction to generate a first reasoning block;

computing a first token-level confidence score for the first reasoning block;

responsive to the first token-level confidence score failing to satisfy a pruning threshold:

discarding the first reasoning block,

obtaining, from the LLM, a first decoding temperature configuration parameter,

configuring the LLM to perform token selection during decoding at a second decoding temperature parameter, wherein the second decoding temperature parameter is greater than the first decoding temperature configuration parameter, and

triggering generating, by the LLM, a second reasoning block after configuring the LLM;

computing a second token-level confidence score for the second reasoning block;

adding the second reasoning block to a reasoning trace responsive to the second token-level confidence score satisfying the pruning threshold; and

generating a provisional answer from the reasoning trace responsive to the second token-level confidence score satisfying a block confidence threshold.

2 . The method of claim 1 , further comprising:

receiving a natural language query from a user application; and

generating the prompt to the LLM from the natural language query.

3 . The method of claim 1 , further comprising:

computing an answer confidence score for the provisional answer; and

responsive to the answer confidence score satisfying an answer confidence threshold, transmitting the provisional answer over a communication network.

4 . The method of claim 1 , further comprising:

computing an answer confidence score for the provisional answer; and

responsive to the answer confidence score failing to satisfy an answer confidence threshold, discarding the provisional answer.

5 . The method of claim 1 , further comprising:

generating, by a decoder of the LLM, the first reasoning block, wherein the first reasoning block is an intermediate reasoning step of the LLM,

wherein the reasoning trace comprises a plurality of reasoning blocks having respective token-level confidence scores satisfying the pruning threshold.

6 . The method of claim 1 , further comprising:

obtaining from a decoder of the LLM, a plurality of tokens demarcated by a plurality of reasoning tokens, wherein the plurality of tokens comprises a natural language utterance;

parsing the plurality of tokens demarcated by the plurality of reasoning tokens to obtain the natural language utterance; and

selecting the natural language utterance as the first reasoning block.

7 . The method of claim 1 , further comprising:

invoking the LLM to generate a predefined number of alternative reasoning blocks, to obtain a plurality of alternative reasoning blocks comprising the second reasoning block.

8 . The method of claim 1 , further comprising:

obtaining, from the LLM, a plurality of alternative reasoning blocks;

computing a plurality of corresponding token-level confidence scores for respective alternative reasoning blocks of the plurality of alternative reasoning blocks; and

selecting an alternative reasoning block with a highest token-level confidence score, of the plurality of corresponding token-level confidence scores, from the plurality of alternative reasoning blocks as the second reasoning block.

9 . The method of claim 1 , wherein the first token-level confidence score is computed as an average log probability of a plurality of tokens of the first reasoning block computed by the LLM.

10 . A method, comprising:

transmitting a prompt to a large language model (LLM), the prompt comprising an instruction to generate a first reasoning block;

computing a first token-level confidence score for the first reasoning block;

iteratively performing, until a maximum number of tokens is generated by the LLM:

invoking the LLM to generate a subsequent reasoning block based on a reasoning trace responsive to a prior token-level confidence score failing to satisfy a pruning threshold, the prior token-level confidence score being the first token-level confidence score in a first iteration, and

adding the subsequent reasoning block to the reasoning trace responsive to a subsequent token-level confidence score of the reasoning block satisfying the pruning threshold and failing to satisfy a block confidence threshold; and

generating a provisional answer from the reasoning trace responsive to a second token-level confidence score satisfying the block confidence threshold.

11 . The method of claim 10 , further comprising:

responsive to the maximum number of tokens being generated by the LLM, invoking the LLM to generate a final answer from the reasoning trace.

12 . A system, comprising:

at least one computer processor;

a large language model (LLM), executing on the at least one computer processor for generating a first reasoning block and a second reasoning block from a natural language query; and

a confidence-aware reasoning (CaR) controller, executing on the at least one computer processor, and configured for:

computing a first token-level confidence score for the first reasoning block,

responsive to the first token-level confidence score failing to satisfy a pruning threshold:

discarding the first reasoning block,

obtaining, from the LLM, a first decoding temperature configuration parameter,

configuring the LLM to perform token selection during decoding at a second decoding temperature parameter, wherein the second decoding temperature parameter is greater than the first decoding temperature configuration parameter, and

triggering generating, by the LLM, a second reasoning block after configuring the LLM,

computing a second token-level confidence score for the second reasoning block,

adding the second reasoning block to a reasoning trace responsive to the second token-level confidence score satisfying the pruning threshold, and

invoking the LLM to generate a provisional answer from the reasoning trace responsive to the second token-level confidence score satisfying a block confidence threshold.

13 . The system of claim 12 , wherein the CaR controller is further configured for:

computing an answer confidence score for the provisional answer,

responsive to the answer confidence score satisfying an answer confidence threshold, transmitting the provisional answer over a communication network, and

responsive to the answer confidence score failing to satisfy the answer confidence threshold, discarding the provisional answer.

14 . The system of claim 12 , wherein the CaR controller is further configured for:

obtaining from a decoder of the LLM, a plurality of tokens demarcated by a plurality of reasoning tokens, wherein the plurality of tokens comprises a natural language utterance;

parsing the plurality of tokens demarcated by the plurality of reasoning tokens to obtain the natural language utterance; and

selecting the natural language utterance as the first reasoning block.

15 . The system of claim 12 , wherein the CaR controller is further configured for:

computing a plurality of corresponding token-level confidence scores for respective alternative reasoning blocks of a plurality of alternative reasoning blocks,

wherein the corresponding token-level confidence scores are computed by the LLM, based on respective average log probabilities of tokens of the respective alternative reasoning blocks; and

selecting an alternative reasoning block with a highest token-level confidence score, of the plurality of corresponding token-level confidence scores, from the plurality of alternative reasoning blocks as the second reasoning block.

16 . The system of claim 12 , wherein the CaR controller is further configured for:

iteratively performing:

invoking the LLM to generate a reasoning block based on the reasoning trace, and

adding the reasoning block to the reasoning trace responsive to a token-level confidence score of the reasoning block satisfying the pruning threshold and failing to satisfy the block confidence threshold.

17 . A method, comprising:

receiving a user query from a user application;

iteratively performing operations comprising:

generating a reasoning block using a large language model (LLM) configured to perform token selection at a first decoding temperature parameter, based on the user query,

computing a token-level confidence score of the reasoning block,

responsive to the token-level confidence score failing to satisfy a pruning threshold:

discarding the reasoning block,

configuring the LLM to perform token selection during decoding at a second decoding temperature parameter, wherein the second decoding temperature parameter is greater than the first decoding temperature parameter,

obtaining, from the LLM, a plurality of alternative reasoning blocks, and

selecting an alternative reasoning block having a highest corresponding token-level confidence score from the plurality of alternative reasoning blocks as the reasoning block, and

appending the reasoning block to a reasoning trace;

generating a response based on the reasoning trace using the LLM; and

transmitting the response over a communication network.

18 . The method of claim 17 , further comprising:

responsive to the token-level confidence score of the reasoning block satisfying a block confidence threshold, performing operations comprising:

invoking the LLM to generate a provisional answer from the reasoning trace,

computing an answer confidence score for the provisional answer, and

responsive to the answer confidence score satisfying an answer confidence threshold, terminating performing the operations.