Interacting with a language model using external knowledge and feedback

1 . A computer-implemented method for interacting with a machine-trained language model, comprising:

receiving an input query;

providing knowledge information based on the input query;

generating original model-input information that includes the input query and the knowledge information, and presenting the original model-input information to the language model;

receiving an original response from the language model, the language model being a generative model trained to auto-regressively predict tokens to follow an initial set of tokens;

programmatically generating a usefulness measure that identifies usefulness of the original response;

in response to determining that the usefulness measure does not satisfy a prescribed test, generating revised model-input information that includes feedback information, presenting the revised model-input information to the language model, and receiving a revised response from the language model in response to the revised model-input information; and

generating and presenting one or more revised instances of model-input information until it is determined that the language model has generated a response that satisfies the prescribed test.

2 . The method of claim 1 , wherein the language model includes weights that are produced in a pre-training operation, and wherein the weights of the language model remain fixed during training of other machine-trained logic used by the method.

3 . The method of claim 1 , wherein the language model includes attention logic for assessing relevance to be given to a part of input information fed to the attention logic when interpreting another part of the input information.

4 . The method of claim 1 , wherein the generating of the revised model-input information is performed upon receiving a user request to generate the revised response.

5 . The method of claim 1 , wherein different actions performed by the method are chosen by a state machine based on state information and a policy, the state information describing aspects of a current dialogue state, and the policy expressing logic for mapping different instances of state information to the different actions.

6 . The method of claim 5 , wherein the state information describes aspects of a current dialogue turn, including at least:

the query;

the knowledge information; and

a last-received response from the language model.

7 . The method of claim 6 , wherein the state information also describes a history of previous dialogue turns, prior to the current dialogue turn.

8 . The method of claim 5 , wherein the policy is chosen to maximize attainment of an objective, and wherein an extent to which an action advances the objective is expressed by a reward signal.

9 . The method of claim 1 , wherein the providing knowledge information comprises:

retrieving initial knowledge information that matches the input query, from one or more knowledge sources;

identifying a chain of evidence based on the initial knowledge; and

validating the chain of evidence, to produce final knowledge information.

10 . The method of claim 1 , wherein the generating of the usefulness measure includes assessing an extent of overlap between the original response and the knowledge information.

11 . The method of claim 1 , further comprising generating the feedback information by retrieving pre-generated prompt information from a data store.

12 . The method of claim 1 , further comprising generating the feedback information using another generative machine-trained model, based on state information that describes aspects of a current dialogue state.

13 . The method of claim 1 , further comprising generating the feedback information using the language model, based on state information that describes aspects of a current dialogue state.

14 . A computing system for interacting with a machine-trained language model, comprising:

computer-readable storage media including:

an instruction data store for storing computer-readable instructions; and

a state data store for storing state information, the state information describing aspects of a current dialogue state; and

a processing system including one or more processors for executing the computer-readable instructions based on the state information in the state data store, to perform operations including:

receiving an input query;

providing knowledge information based on the input query;

generating original model-input information including the input query and the knowledge information, and presenting the original model-input information to the language model;

receiving an original response from the language model, the language model being a generative model trained to auto-regressively predict tokens to follow an initial set of tokens;

programmatically generating a usefulness measure that identifies usefulness of the original response;

in response to determining that the usefulness measure does not satisfy a prescribed test, generating revised model-input information that includes feedback information, presenting the revised model-input information to the language model, and receiving a revised response from the language model in response to the revised model-input information; and

generating and presenting one or more revised instances of model-input information until it is determined that the language model has generated a response that satisfies the prescribed test.

15 . The computing system of claim 14 , wherein the processing system implements a state machine for performing different actions based on the state information and a policy, the policy expressing logic for mapping different instances of state information to the different actions.

16 . The computing system of claim 14 , wherein the operations further include generating the feedback information by retrieving pre-generated prompt information from a data store.

17 . The computing system of claim 14 , wherein the operations further include generating the feedback information using another generative machine-trained model, based on the state information.

18 . The computing system of claim 14 , wherein the operations further include generating the feedback information using the language model, based on the state information.

19 . A computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer-readable instructions to perform operations, the operations comprising:

receiving an input query;

providing knowledge information based on the input query;

generating original model-input information that includes the input query and the knowledge information, and presenting the original model-input information to machine-trained a language model;

receiving an original response from the language model, the language model being a generative model trained to auto-regressively predict tokens to follow an initial set of tokens;

programmatically generating a usefulness measure that identifies usefulness of the original response;

in response to determining that the usefulness measure does not satisfy a prescribed test, generating feedback information;

generating revised model-input information that includes the feedback information;

presenting the revised model-input information to the language model;

receiving a revised response from the language model in response to the revised model-input information; and

generating and presenting one or more revised instances of model-input information until it is determined that the language model has generated a response that satisfies the prescribed test.

20 . The computer-readable storage medium of claim 19 , wherein the operations further include generating the feedback information using the language model, based on state information.