Selection-inference neural network systems
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a response to a query input using a selection-inference neural network.
1 . A method performed by one or more computers, the method comprising:
obtaining a context input comprising context information, the context information comprising one or more natural language statements that each represent a fact or a rule relating to an environment, wherein the environment is a real world environment and the method is used for controlling an agent acting in the real world environment to perform a task;
receiving a query input comprising a query relating to the environment, wherein the query input relates to an action to be performed by the agent;
generating a response to the query input by performing update iterations until a termination criterion is satisfied, the generating comprising, at each update iteration:
processing a selection input comprising the context input and the query input using a selection neural network to generate a selection output for the update iteration that comprises one or more of the natural language statements from the context input; and
processing an inference input comprising the selection output for the update iteration using an inference neural network to generate an inference output that comprises a natural language statement that represents a new fact for the update iteration; and
at each update iteration other than a last update iteration:
updating the context input to include the natural language statement in the inference output for the update iteration; and
evaluating whether the response to the query input indicates that a proposed action violates a rule relating to control of the agent; and
using the response to the query input to control the agent in the real world environment.
2 . The method of claim 1 , further comprising:
providing, as the response, (i) a natural language output derived from the natural language statement in the inference output for the last update iteration and (ii) a reasoning trace that comprises the natural language statement in the inference output at each update iteration other than the last update iteration.
3 . The method of claim 2 , wherein the reasoning trace further comprises the selection outputs for the update iterations.
4 . The method of claim 1 , wherein the inference input does not include the context input or the query input.
5 . The method of claim 1 wherein the agent is a mechanical agent acting in the real world environment to perform the task.
6 . The method of claim 5 , wherein
obtaining the context input comprises obtaining, from one or more sensors, one or more observations of the real world environment, and processing the one or more observations to generate a natural language representation of the one or more observations.
7 . The method of claim 6 , further comprising:
using the natural language representation of the one or more observations to provide one or more of the natural language statements of the context information.
8 . The method of claim 6 wherein the mechanical agent has an agent control system to control actions of the mechanical agent, wherein the query input comprises one or more natural language queries, and wherein receiving the query input comprises:
receiving a control signal from the agent control system; and
generating the one or more natural language queries from the control signal.
9 . The method of claim 6 wherein the mechanical agent comprises an autonomous or semi-autonomous vehicle navigating in the real-world environment, and wherein the action comprises an action to control movement of the vehicle in the real-world environment.
10 . The method of claim 1 , wherein:
the context input is derived from at least an observation characterizing a current state of the real-world environment that is generated from measurements from one or more sensors configured to sense the real-world environment,
the query input comprises data characterizing planned navigation of an agent, and
the response to the query input characterizes an action to be performed by the agent in response to the observation.
11 . The method of claim 10 , wherein the agent is a robot or an autonomous vehicle.
12 . The method of claim 10 , further comprising:
controlling navigation of the agent based on the response to the query input.
13 . The method of claim 1 wherein the environment is a manufacturing plant for manufacturing a product, the manufacturing plant comprising a plurality of manufacturing units configured such that an intermediate version or component of the product is moveable between the manufacturing units during manufacture of the product, wherein the method is used for controlling one or more of the manufacturing units or for controlling movement of the intermediate version or component of the product between the manufacturing units, and wherein the agent is a plant control system of the manufacturing plant to control the manufacturing units or to control the movement; wherein
obtaining the context input comprises obtaining, from one or more sensors, one or more observations of the manufacturing units or of the movement, and processing the one or more observations to generate a natural language representation of the one or more observations; and wherein
the query input relates to an action that controls operation of one or more of the manufacturing units or that controls the movement;
the method further comprising:
using the natural language representation of the one or more observations to provide one or more of the natural language statements of the context information; and
using the response to the query input to control operation of one or more of the manufacturing units or to control the movement.
14 . The method of claim 13 wherein the query input comprises one or more natural language queries, and wherein receiving the query input comprises:
receiving a control signal from the plant control system; and
generating the one or more natural language queries from the control signal.
15 . The method of claim 1 wherein the query input comprises a natural language description that defines information the response is to provide.
16 . The method of claim 1 , further comprising:
generating at least one of the natural language statements in the context input by processing an observation using a first machine learning model configured to process the observation to generate natural language text describing the observation.
17 . The method of claim 16 , further comprising:
generating at least one of the natural language statements in the context input by processing the observation using a second machine learning model configured to process (i) the observation, (ii) the natural language text describing the observation, or (iii) both to generate natural language text characterizing one or more rules for determining the new fact that are relevant to the observation.
18 . The method of claim 1 , wherein the termination criterion is satisfied when a threshold number of update iterations have been performed.
19 . The method of claim 1 , wherein processing a selection input comprising the context input and the query input using a selection neural network to generate a selection output for the update iteration that comprises one or more of the natural language statements in the context information comprises:
selecting a respective natural language statement from the context information at each of a sequence of one or more selection iterations, the selecting comprising, at each selection iteration:
generating an input for the selection iteration, the input comprising the context input, the query input, and any natural language statements selected at any previous selection iterations that precede the selection iteration in the sequence; and
selecting the respective natural language statement by processing the input for the selection iteration using the selection neural network.
20 . The method of claim 19 , wherein selecting the respective natural language statement of the context information for the selection iteration comprises:
for each of a set of natural language statements in the context information:
processing the input for the selection iteration using the selection neural network to determine a likelihood assigned to the natural language statement by the selection neural network; and
selecting, from the set of natural language statements, the natural language statement with a highest likelihood.
21 . The method of claim 1 , wherein:
the selection neural network and the inference neural network are the same neural network,
the selection input comprises a first few-shot prompt, and
the inference input comprises a second, different few-shot prompt.
22 . The method of claim 21 , wherein:
generating an input for the selection iteration comprises modifying the first few-shot prompt to include the natural language statements selected at any previous selection iterations that precede the selection iteration in a sequence of one or more selection iterations.
23 . The method of claim 1 , wherein the selection neural network and the inference neural network are the same pre-trained language model neural network.
24 . One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
obtaining a context input comprising context information, the context information comprising one or more natural language statements that each represent a fact or a rule relating to an environment, wherein the environment is a real world environment and the method is used for controlling an agent acting in the real world environment to perform a task;
receiving a query input comprising a query relating to the environment, wherein the query input relates to an action to be performed by the agent;
generating a response to the query input by performing update iterations until a termination criterion is satisfied, the generating comprising, at each update iteration:
processing a selection input comprising the context input and the query input using a selection neural network to generate a selection output for the update iteration that comprises one or more of the natural language statements from the context input; and
processing an inference input comprising the selection output for the update iteration using an inference neural network to generate an inference output that comprises a natural language statement that represents a new fact for the update iteration; and
at each update iteration other than a last update iteration:
updating the context input to include the natural language statement in the inference output for the update iteration; and
evaluating whether the response to the query input indicates that a proposed action violates a rule relating to control of the agent; and
using the response to the query input to control the agent in the real world environment.
25 . A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
obtaining a context input comprising context information, the context information comprising one or more natural language statements that each represent a fact or a rule relating to an environment, wherein the environment is a real world environment and the method is used for controlling an agent acting in the real world environment to perform a task;
receiving a query input comprising a query relating to the environment, wherein the query input relates to an action to be performed by the agent;
generating a response to the query input by performing update iterations until a termination criterion is satisfied, the generating comprising, at each update iteration:
processing a selection input comprising the context input and the query input using a selection neural network to generate a selection output for the update iteration that comprises one or more of the natural language statements from the context input; and
processing an inference input comprising the selection output for the update iteration using an inference neural network to generate an inference output that comprises a natural language statement that represents a new fact for the update iteration; and
at each update iteration other than a last update iteration:
updating the context input to include the natural language statement in the inference output for the update iteration; and
evaluating whether the response to the query input indicates that a proposed action violates a rule relating to control of the agent; and
using the response to the query input to control the agent in the real world environment.
26 . The system of claim 25 , the operations further comprising:
providing, as the response, (i) a natural language output derived from the natural language statement in the inference output for the last update iteration and (ii) a reasoning trace that comprises the natural language statement in the inference output at each update iteration other than the last update iteration.