Augmenting neural networks with external memory

1. An augmented neural network system for processing a sequence of system inputs to generate a sequence of system outputs, the augmented neural network system comprising one or more computers storing instructions that when executed by the one or more computers cause the one or more computers to implement:

a controller neural network configured to receive a neural network input at each of a plurality of time steps and to process the neural network input to generate a neural network output for the time step, wherein each neural network output includes:

a read key, and

a write vector;

an external memory that is external to the controller neural network; and

a Least Recently Used Access (LRUA) subsystem that is configured to:

maintain a respective usage weight for each of a plurality of locations in the external memory that, for each location, is based on reading weights and writing weights generated for the location by the LRUA subsystem across the plurality of time steps, and

for each of the plurality of time steps:

generate a respective reading weight for the time step for each of the plurality of locations in the external memory using the read key,

read data from the plurality of locations in the external memory in accordance with the reading weights,

generate a respective writing weight for the time step for each of the plurality of locations in the external memory from a respective reading weight for the location generated at a preceding time step and a respective usage weight for the location after the preceding time step,

write the write vector to the plurality of locations in the external memory in accordance with the writing weights, and

update the respective usage weight after the preceding time step for each of the plurality of locations in the external memory from the respective reading weight for the location for the time step and the respective writing weight for the location for the time step to generate an updated usage weight for a next time step, comprising, for each of the plurality of locations:

determining a decayed usage weight for the location from the usage weight for the location after the preceding time step; and

determining the updated usage weight for the location for the next time step by summing the decayed usage weight, the reading weight for the location for the time step, and the writing weight for the location for the time step.

2. The system of claim 1 , wherein the read key is different from the write vector.

3. The system of claim 1 , wherein the read key is the same as the write vector.

4. The system of claim 1 , wherein the LRUA subsystem is further configured to, for each of the plurality of time steps:

prior to writing the write vector to the plurality of locations and updating the respective usage weight for each of the plurality of locations:

identify a location in the external memory that has a lowest usage weight; and

erase the data stored in the identified location.

5. The system of claim 4 , wherein the instructions cause one or more computers to implement one or more output layers, and wherein the one or more output layers are configured to, for each of the plurality of time steps:

process the data read from the external memory for the time step to generate a system output for the time step.

6. The system of claim 1 , wherein the neural network output for each of the plurality of time steps further includes a system output portion.

7. The system of claim 6 , wherein the LRUA subsystem is further configured to, for each of the plurality of time steps, provide an output derived from the system output portion as a system output for the time step.

8. The system claim 7 , wherein the instructions cause one or more computers to implement one or more output layers, and wherein the one or more output layers are configured to, for each of the plurality of time steps:

process the data read from the external memory and the system output portion for the time step to generate a system output for the time step.

9. The system of claim 1 , wherein the neural network input for each time step is a respective system input from the sequence of system inputs.

10. The system of claim 1 , wherein the LRUA subsystem is further configured to, for each of the plurality of time steps:

combine the data read from the external memory and a respective system input from the sequence of system inputs to generate a subsequent neural network input for a subsequent time step.

11. The system of claim 10 , wherein generating a respective writing weight for each of the plurality of locations in the external memory from a respective reading weight for the location and a preceding time step and the respective usage weight for the location comprises: determining a convex combination of the respective reading weight for the location and the preceding time step and the respective usage weight for the location in accordance with an interpolation parameter.

12. The system of claim 1 , wherein determining the decayed usage weight for the location from the usage weight for the location after the preceding time step comprises:

applying a decay parameter to the usage weight for the location after the preceding time step.

13. The system of claim 1 , wherein the controller neural network has been trained using a meta-learning training technique.

14. The system of claim 1 , wherein:

generating a respective writing weight for the time step for each of the plurality of locations in the external memory from a respective reading weight for the location generated at a preceding time step and a respective usage weight for the location after the preceding time step comprises:

determining a respective least-used weight after the preceding time step for each of the plurality of locations from the respective usage weights for the plurality of locations after the preceding time step; and

combining, for each of the locations, the respective reading weight for the location generated at the preceding time step and the respective least-used weight for the location after the preceding time step to generate the respective writing weight for the time step for the location.

15. The system of claim 14 , wherein determining the respective least-used weight after the preceding time step for each of the plurality of locations from the respective usage weights for the plurality of locations after the preceding time step comprises:

setting the least-used weight for a location that has a smallest usage weight after the preceding time step to one; and

setting the least-used weights for all locations other than the location that has the smallest usage weight after the preceding time step to zero.

16. A computer-implemented method for processing a sequence of system inputs to generate a sequence of system outputs, the method comprising:

maintaining a respective usage weight for each of a plurality of locations in an external memory that, for each location, is based on reading weights and writing weights generated for the location across a plurality of time step;

receiving a neural network input at each of the plurality of time steps;

processing the neural network input using a controller neural network to generate a neural network output for the time step, wherein the neural network output includes a read key and a write vector, wherein the external memory is external to the neural network;

generating a respective reading weight for the time step for each of the plurality of locations in the external memory using the read key;

reading data from the plurality of locations in the external memory in accordance with the reading weights;

generating a respective writing weight for the time step for each of the plurality of locations in the external memory from a respective reading weight for the location generated at a preceding time step and a respective usage weight for the location after the preceding time step;

writing the write vector to the plurality of locations in the external memory in accordance with the writing weights; and

updating the respective usage weight after the preceding time step for each of the plurality of locations in the external memory from the respective reading weight for the location for the time step and the respective writing weight for the location for the time step to generate an updated usage weight for a next time step, comprising, for each of the plurality of locations:

determining a decayed usage weight for the location from the usage weight for the location after the preceding time step; and

determining the updated usage weight for the location for the next time step by summing the decayed usage weight, the reading weight for the location for the time step, and the writing weight for the location for the time step.

17. The method of claim 16 , further comprising:

prior to writing the write vector to the plurality of locations and updating the respective usage weight for each of the plurality of locations:

identifying a location in the external memory that has a lowest usage weight; and

erasing the data stored in the identified location.

18. The method of claim 16 , further comprising:

processing the data read from the external memory through one or more output neural network layers to generate a system output.

19. The method of claim 16 , wherein the neural network input is a respective system input from the sequence of system inputs.

20. The method of claim 16 , further comprising: combining the data read from the external memory and a system input from the sequence of system inputs to generate a subsequent neural network input for a subsequent time step.

21. The method of claim 16 , wherein generating a respective writing weight for each of the plurality of locations in the external memory from a respective reading weight for the location and a preceding time step and the respective usage weight for the location comprises: determining a convex combination of the respective reading weight for the location and the preceding time step and the respective usage weight for the location in accordance with an interpolation parameter.

22. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations for generating a sequence of outputs from a sequence of inputs, the operations comprising:

maintaining a respective usage weight for each of a plurality of locations in an external memory that, for each location, is based on reading weights and writing weights generated for the location across a plurality of time steps;

receiving a neural network input at each of the plurality of time steps;

processing the neural network input using a controller neural network to generate a neural network output for the time step, wherein the neural network output includes a read key and a write vector, wherein the external memory is external to the neural network;

generating a respective reading weight for the time step for each of the plurality of locations in the external memory using the read key;

reading data from the plurality of locations in the external memory in accordance with the reading weights;

generating a respective writing weight for the time step for each of the plurality of locations in the external memory from a respective reading weight for the location generated at a preceding time step and a respective usage weight for the location after the preceding time step;

writing the write vector to the plurality of locations in the external memory in accordance with the writing weights; and

updating the respective usage weight after the preceding time step for each of the plurality of locations in the external memory from the respective reading weight for the location for the time step and the respective writing weight for the location for the time step to generate an updated usage weight for a next time step, comprising, for each of the plurality of locations:

determining a decayed usage weight for the location from the usage weight for the location after the preceding time step; and

determining the updated usage weight for the location for the next time step by summing the decayed usage weight, the reading weight for the location for the time step and the writing weight for the location for the time step.