Granted Patent B2

US 12,572,782 · App. 18/662,972 · Granted Mar 10, 2026

Inventors: Jack William Rae (London, GB); Timothy Paul Lillicrap (London, GB); Sergey Bartunov (London, GB)

Assignee: GDM Holding LLC

G06N3/045G06F16/2272G06N3/08

Abstract

Claims (57)

1 . A method for training a neural network memory system having neural network memory system parameters, the method comprising:

receiving a training dataset comprising a plurality of sets of data items, wherein each set of data items in the plurality of sets of data items corresponds to a different respective task;

at each training iteration of a plurality of training iterations:

selecting, from the plurality of sets of data items, an input set of data items for the training iteration, wherein the input set of data items corresponds to a respective task for the training iteration;

storing data representing the input set of data items for the training iteration at a plurality of memory locations in a memory;

generating a plurality of training examples, wherein each training example comprises (i) a respective training data item and (ii) a respective label indicating whether or not the training data item belongs to the input set of data items for the training iteration;

for each training example, processing the respective training data item in the training example using the neural network memory system in accordance with current values of the neural network memory system parameters to generate, based on a weighting associated with the plurality of memory locations and the data representing the input set of data items stored at the plurality of memory locations, a respective training output for the training example indicating whether or not the respective training data item belongs to the input set of data items; and

training the neural network memory system using the respective labels in the plurality of training examples and the respective training outputs to minimize an objective function.

2 . The method of claim 1 , wherein training the neural network memory system using the respective labels in the plurality of training examples and the respective training outputs to minimize an objective function comprises:

updating the neural network memory system parameters using gradients of the objective function with respect to the neural network memory system parameters.

3 . The method of claim 2 , wherein updating the neural network memory system parameters using gradients of the objective function with respect to the neural network memory system parameters comprises:

computing the gradients of the objective function with respect to the neural network memory system parameters; and

backpropagating the gradients of the objective function into the neural network memory system to adjust values of the parameters of the neural network memory system.

4 . The method of claim 1 , wherein the objective function comprises a cross-entropy loss term.

5 . The method of claim 1 , wherein generating the plurality of training examples comprises sampling a predetermined number of data items from the input set of data items.

6 . The method of claim 1 , wherein generating the plurality of training examples comprises sampling a predetermined number of data items from one or more sets of data items in the training dataset that are not the input set of data items.

7 . The method of claim 1 , wherein for each training example, processing the respective training data item in the training example using the neural network memory system in accordance with current values of the neural network memory system parameters to generate, based on a weighting associated with the plurality of memory locations and the data representing the input set of data items stored at the plurality of memory locations, a respective training output for the training example indicating whether or not the respective training data item belongs to the input set of data items comprises:

processing, by a query neural network, the training data item to generate a training query.

8 . The method of claim 7 , wherein processing, by a query neural network, the training data item to generate a training query comprises:

processing, by an encoder neural network, the training data item to generate a representation of the training data item; and

processing, by the query neural network, the representation of the training data item to generate the training query.

9 . The method of claim 7 , further comprising:

generating, from (i) the training query and (ii) a respective key for each of the plurality of memory locations of the memory, a respective weight for each of the plurality of memory locations in the memory;

generating output memory data from the memory based on the respective weights for the plurality of memory locations and data stored at the plurality of memory locations; and

generating, from at least the output memory data, an output that indicates whether or not the training data item belongs to the input set of data items.

10 . The method of claim 9 , wherein generating output memory data from the memory based on the respective weights for the plurality of memory locations and data stored at the plurality of memory locations comprises:

computing, for each memory location, a multiplication between the data stored at the memory location and the respective weight for the memory location.

11 . The method of claim 9 , wherein generating, from at least the output memory data, an output that indicates whether or not the training data item belongs to the input set of data items comprises:

processing, by an output representation neural network, at least the output memory data to generate the output that indicates whether or not the training data item belongs to the input set of data items.

12 . The method of claim 11 , further comprising:

processing a representation of the training data item using a write word neural network to generate write data; and wherein

processing, by an output representation neural network, at least the output memory data to generate an output that indicates whether or not the training data item belongs to the input set of data items comprises, processing, by the output representation neural network, at least the write data and the output memory data.

13 . The method of claim 12 , wherein processing, by an output representation neural network, at least the output memory data to generate an output that indicates whether or not the training data item belongs to the input set of data items comprises, processing, by the output representation neural network, the write data, the representation of the training data item and the output memory data.

14 . The method of claim 9 , wherein generating, from (i) the training query and (ii) a respective key for each of a plurality of memory locations of a memory, a respective weight for each of the plurality of memory locations in the memory comprises:

generating, from the training query, a decorrelated query; and

generating, from (i) the decorrelated query and (ii) the respective key for each of a plurality of memory locations of a memory, the respective weight for each of the plurality of memory locations in the memory.

15 . The method of claim 9 , wherein the respective weights are sparse weights that are only non-zero for a proper subset of the memory locations in the memory.

16 . The method of claim 9 , further comprising initializing the neural network memory system with key data prior to the training, wherein the key data remains fixed throughout the training.

17 . A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training a neural network memory system having neural network memory system parameters, the operations comprising:

receiving a training dataset comprising a plurality of sets of data items, wherein each set of data items in the plurality of sets of data items corresponds to a different respective task;

at each training iteration of a plurality of training iterations:

selecting, from the plurality of sets of data items, an input set of data items for the training iteration, wherein the input set of data items corresponds to a respective task for the training iteration;

encoding data representing the input set of data items in a memory for the training iteration at a plurality of memory locations;

generating a plurality of training examples, wherein each training example comprises (i) a respective training data item and (ii) a respective label indicating whether or not the training data item belongs to the input set of data items for the training iteration;

for each training example, processing the respective training data item in the training example using the neural network memory system in accordance with current values of the neural network memory system parameters to generate, based on a weighting associated with the plurality of memory locations and the data representing the input set of data items stored at the plurality of memory locations, a respective training output for the training example indicating whether or not the respective training data item belongs to the input set of data items; and

training the neural network memory system using the respective labels in the plurality of training examples and the respective training outputs to minimize an objective function.

18 . The system of claim 17 , wherein training the neural network memory system using the respective labels in the plurality of training examples and the respective training outputs to minimize an objective function comprises:

updating the neural network memory system parameters using gradients of the objective function with respect to the neural network memory system parameters.

19 . The system of claim 17 , wherein the objective function comprises a cross-entropy loss term.

20 . One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training a neural network memory system having neural network memory system parameters, the operations comprising:

receiving a training dataset comprising a plurality of sets of data items, wherein each set of data items in the plurality of sets of data items corresponds to a different respective task;

at each training iteration of a plurality of training iterations:

selecting, from the plurality of sets of data items, an input set of data items for the training iteration, wherein the input set of data items corresponds to a respective task for the training iteration;

encoding data representing the input set of data items in a memory for the training iteration at a plurality of memory locations;

generating a plurality of training examples, wherein each training example comprises (i) a respective training data item and (ii) a respective label indicating whether or not the training data item belongs to the input set of data items for the training iteration;

for each training example, processing the respective training data item in the training example using the neural network memory system in accordance with current values of the neural network memory system parameters to generate, based on a weighting associated with the plurality of memory locations and the data representing the input set of data items stored at the plurality of memory locations, a respective training output for the training example indicating whether or not the respective training data item belongs to the input set of data items; and

training the neural network memory system using the respective labels in the plurality of training examples and the respective training outputs to minimize an objective function.

Assignments (2)

Continuity (4)

References Cited (80)