Image depth prediction neural networks

1. A system comprising:

an image depth prediction neural network implemented by one or more computers, wherein the image depth prediction neural network is a recurrent neural network that is configured to receive a sequence of images and, for each image in the sequence:

process the image, which is a current image at a first time step in the sequence, in accordance with a current internal state of the recurrent neural network to (i) update the current internal state and (ii) generate a depth output that characterizes a predicted depth of a future image at a second, subsequent time step in the sequence, wherein the future image is an image subsequent to the current image in the sequence, wherein the depth output comprises a set of values defining the topology of a scene represented by the future image in a third, depth dimension; and

an image generation subsystem configured to, for each image in the sequence:

receive the depth output that characterizes the predicted depth of the future image, and

generate an image frame that is a prediction of the future image using the depth output of the future image.

2. The system of claim 1 , wherein the depth output comprises a predicted depth value for each pixel of a plurality of pixels in the future image that represents a respective distance of a scene depicted at the pixel from a focal plane of the future image.

3. The system of claim 1 , wherein the future image immediately follows the image in the sequence of images.

4. The system of claim 1 , wherein the sequence of images are frames of video captured by a camera of a robotic agent.

5. The system of claim 1 , wherein the image depth prediction neural network comprises one or more convolutional long short-term memory (LSTM) neural network layers.

6. The system of claim 1 , wherein the image depth prediction neural network comprises one or more feedforward convolutional neural network layers.

7. The system of claim 1 , wherein the image depth prediction neural network comprises:

a down-sampling recurrent sub-neural network followed by an up-sampling recurrent sub-neural network,

wherein the down-sampling recurrent sub-neural network is configured to, for each image in the sequence:

process the image to generate a down-sampled output having a lower resolution than the image, and

wherein the up-sampling recurrent sub-neural network is configured to, for each image in the sequence:

process the down-sampled output for the image to generate the depth output.

8. A method comprising:

receiving a sequence of images; and

for each image in the sequence:

processing the image using an image depth prediction neural network,

wherein the image depth prediction neural network is a recurrent neural network that is configured to, for each image in the sequence:

process the image, which is a current image at a first time step in the sequence, in accordance with a current internal state of the recurrent neural network to (i) update the current internal state and (ii) generate a depth output that characterizes a predicted depth of a future image at a second, subsequent time step in the sequence, wherein the future image is an image subsequent to the current image in the sequence;

receiving the depth output that characterizes the predicted depth of the future image; and

generating an image frame that is a prediction of the future image using the depth output of the future image,

wherein the depth output comprises a set of values defining the topology of a scene represented by the images in a third, depth dimension.

9. The method of claim 8 , wherein the depth output comprises a predicted depth value for each pixel of a plurality of pixels in the future image that represents a respective distance of a scene depicted at the pixel from a focal plane of the future image.

10. The method of claim 8 , wherein the future image immediately follows the image in the sequence of images.

11. The method of claim 8 , wherein the sequence of images are frames of video captured by a camera of a robotic agent.

12. The method of claim 8 , wherein the image depth prediction neural network comprises one or more convolutional long short-term memory (LSTM) neural network layers.

13. The method of claim 8 , wherein the image depth prediction neural network comprises one or more feedforward convolutional neural network layers.

14. The method of claim 8 , wherein the image depth prediction neural network comprises:

a down-sampling recurrent sub-neural network followed by an up-sampling recurrent sub-neural network,

wherein the down-sampling recurrent sub-neural network is configured to, for each image in the sequence:

process the image to generate a down-sampled output having a lower resolution than the image, and

wherein the up-sampling recurrent sub-neural network is configured to, for each image in the sequence:

process the down-sampled output for the image to generate the depth output.

15. One or more non-transitory computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

receiving a sequence of images; and

for each image in the sequence:

processing the image using an image depth prediction neural network,

wherein the image depth prediction neural network is a recurrent neural network that is configured to, for each image in the sequence:

process the image, which is a current image at a first time step in the sequence, in accordance with a current internal state of the recurrent neural network to (i) update the current internal state and (ii) generate a depth output that characterizes a predicted depth of a future image at a second, subsequent time step in the sequence, wherein the future image is an image subsequent to the current image in the sequence;

receiving the depth output that characterizes the predicted depth of the future image; and

generating an image frame that is a prediction of the future image using the depth output of the future image,

wherein the depth output comprises a set of values defining the topology of a scene represented by the images in a third, depth dimension.

16. The one or more non-transitory computer storage media of claim 15 , wherein the depth output comprises a predicted depth value for each pixel of a plurality of pixels in the future image that represents a respective distance of a scene depicted at the pixel from a focal plane of the future image.

17. The one or more non-transitory computer storage media of claim 15 , wherein the future image immediately follows the image in the sequence of images.

18. The one or more non-transitory computer storage media of claim 15 , wherein the image depth prediction neural network comprises one or more convolutional long short-term memory (LSTM) neural network layers.

19. The one or more non-transitory computer storage media of claim 15 , wherein the image depth prediction neural network comprises one or more feedforward convolutional neural network layers.

20. The one or more non-transitory computer storage media of claim 15 , wherein the image depth prediction neural network comprises:

a down-sampling recurrent sub-neural network followed by an up-sampling recurrent sub-neural network,

wherein the down-sampling recurrent sub-neural network is configured to, for each image in the sequence:

process the image to generate a down-sampled output having a lower resolution than the image, and

wherein the up-sampling recurrent sub-neural network is configured to, for each image in the sequence:

process the down-sampled output for the image to generate the depth output.