Memory augmented generative temporal models

1. A method performed by one or more data processing apparatus, the method comprising, for each of a plurality of time steps:

receiving a controller neural network input for the time step;

processing the controller neural network input for the time step using a controller neural network, in accordance with a current hidden state of the controller neural network, to generate an updated hidden state of the controller neural network for the time step;

generating a memory context vector by reading data from an external memory using the updated hidden state of the controller neural network for the time step;

determining, from at least the memory context vector generated by reading data from the external memory using the updated hidden state of the controller neural network for the time step, a set of latent variables for the time step that collectively define a compressed representation of a predicted observation characterizing the environment at the time step, comprising:

receiving an actual observation characterizing an environment at the time step;

processing: (i) the memory context vector generated by reading data from the external memory using the updated hidden state of the controller neural network for the time step, and (ii) the actual observation, using a posterior map neural network to generate parameters of a posterior distribution over possible latent variable values; and

sampling the latent variables from the posterior distribution in accordance with the parameters of the posterior distribution;

generating the predicted observation characterizing the environment at the time step using at least the set of latent variables.

2. The method of claim 1 , wherein the controller neural network input for the time step comprises a set of latent variables generated at a previous time step that collectively define a compressed representation of a predicted observation characterized the environment at the previous time step.

3. The method of claim 1 , further comprising writing data to the external memory using the set of latent variables, the updated hidden state, or both, comprising:

writing the set of latent variables for the time step to the external memory.

4. The method of claim 3 , wherein writing the set of latent variables for the time step to the external memory comprises:

writing the set of latent variables for the time step to a specified location in the external memory.

5. The method of claim 3 , wherein writing data to the external memory comprises:

generating a set of writing weights, wherein the set of writing weights includes a respective writing weight for each of a plurality of locations in the external memory; and

writing data defined by the updated hidden state to the locations in the external memory in accordance with the writing weights.

6. The method of claim 1 , wherein generating a memory context vector by reading data from the external memory using the updated hidden state comprises, for each of one or more read heads:

generating a set of reading weights for the read head from the updated hidden state, wherein the set of reading weights includes a respective reading weight for each of a plurality of locations in the external memory; and

reading data from the plurality of locations in the external memory in accordance with the reading weights.

7. The method of claim 6 , wherein the reading weights are position-based addressing weights.

8. The method of claim 6 , wherein the reading weights are content-based addressing weights.

9. The method of claim 6 , wherein the reading weights are a combination of position-based addressing weights and content-based addressing weights.

10. The method of claim 1 , further comprising generating a controller neural network input for the next time step, comprising:

combining the latent variables for the time step and an external context vector to generate the controller neural network input for the next time step.

11. The method of claim 1 , further comprising generating a controller neural network input for the next time step, comprising:

combining the latent variables for the time step and data from the external memory to generate the controller neural network input for the next time step.

12. The method of claim 1 , further comprising generating a controller neural network input for the next time step, comprising:

combining the latent variables for the time step, data read from the external memory, and an external context vector to generate the controller neural network input for the next time step.

13. The method of claim 1 , wherein the controller neural network input for the time step comprises latent variables for a preceding time step and an external context vector for the time step.

14. The method of claim 13 , wherein the external context vector represents one or more external variables that influence characteristics of observations of the environment.

15. The method of claim 1 , wherein for each of the plurality of time steps, generating the memory context vector by reading data from the external memory using the updated hidden state of the controller neural network for the time step comprises:

modifying the memory context vector using correction biases generated by processing the updated hidden state of the controller neural network for the time step using a correction bias neural network.

16. The method of claim 1 , wherein the controller neural network and the posterior map neural network have been jointly trained to optimize an objective function that comprises, for each of the plurality of time steps, a divergence measure between: (i) the posterior distribution generated at the time step, and (ii) a prior distribution at the time step.

17. A system comprising:

one or more computers; and

one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising, for each of a plurality of time steps:

receiving a controller neural network input for the time step;

processing the controller neural network input for the time step using a controller neural network, in accordance with a current hidden state of the controller neural network, to generate an updated hidden state of the controller neural network for the time step;

generating a memory context vector by reading data from an external memory using the updated hidden state of the controller neural network for the time step;

determining, from at least the memory context vector generated by reading data from the external memory using the updated hidden state of the controller neural network for the time step, a set of latent variables for the time step that collectively define a compressed representation of a predicted observation characterizing the environment at the time step, comprising:

receiving an actual observation characterizing an environment at the time step;

processing: (i) the memory context vector generated by reading data from the external memory using the updated hidden state of the controller neural network for the time step, and (ii) the actual observation, using a posterior map neural network to generate parameters of a posterior distribution over possible latent variable values; and

sampling the latent variables from the posterior distribution in accordance with the parameters of the posterior distribution;

generating the predicted observation characterizing the environment at the time step using at least the set of latent variables.

18. The system of claim 17 , wherein the controller neural network input for the time step comprises a set of latent variables generated at a previous time step that collectively define a compressed representation of a predicted observation characterized the environment at the previous time step.

19. The system of claim 17 , further comprising writing data to the external memory using the set of latent variables, the updated hidden state, or both, comprising:

writing the set of latent variables for the time step to the external memory.

20. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising, for each of a plurality of time steps:

receiving a controller neural network input for the time step;

processing the controller neural network input for the time step using a controller neural network, in accordance with a current hidden state of the controller neural network, to generate an updated hidden state of the controller neural network for the time step;

generating a memory context vector by reading data from an external memory using the updated hidden state of the controller neural network for the time step;

determining, from at least the memory context vector generated by reading data from the external memory using the updated hidden state of the controller neural network for the time step, a set of latent variables for the time step that collectively define a compressed representation of a predicted observation characterizing the environment at the time step, comprising:

receiving an actual observation characterizing an environment at the time step;

processing: (i) the memory context vector generated by reading data from the external memory using the updated hidden state of the controller neural network for the time step, and (ii) the actual observation, using a posterior map neural network to generate parameters of a posterior distribution over possible latent variable values; and

sampling the latent variables from the posterior distribution in accordance with the parameters of the posterior distribution;

generating the predicted observation characterizing the environment at the time step using at least the set of latent variables.