Granted Patent
B2
US 12,718,053 · App. 18/640,741 · Granted Aug 25, 2026
Learning observation representations by predicting the future in latent space
View Patent ↗
Loading inventors, assignments & file history…
Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST
Recorded Jun 6, 2025
From: DEEPMIND TECHNOLOGIES LIMITED
To: GDM HOLDING LLC
Reel/Frame 071498/0210 →
ASSIGNMENT OF ASSIGNOR'S INTEREST
Recorded May 8, 2024
From: VAN DEN OORD, AARON GERARD ANTONIUS; LI, YAZHE; VINYALS, ORIOL
To: DEEPMIND TECHNOLOGIES LIMITED
Reel/Frame 067346/0467 →
Continuity (4)
Provisional Application
62737878
· Sep 27, 2018
References Cited (164)
US 6516021B1
· Abbott
· 2003
[cited by applicant]
US 9135241B2
· Bangalore
· 2015
[cited by examiner]
US 20050114278A1
· Saptharishi
· 2005
[cited by examiner]
US 20080040660A1
· Georke
· 2008
[cited by examiner]
US 20080172348A1
· Tien
· 2008
[cited by examiner]
US 20140025613A1
· Ponulak
· 2014
[cited by examiner]
US 20200174490A1
· Ogale et al.
· 2020
[cited by applicant]
US 20210004677A1
· Menick et al.
· 2021
[cited by applicant]
US 20210287297A1
· Hayward et al.
· 2021
[cited by applicant]
US 20220076099A1
· Sernnanet et al.
· 2022
[cited by applicant]
Abbeel et al., “Apprenticeship learning via inverse reinforcement learning,” Proceedings of the twenty-first international conference on Machine learning, Jul. 2044, 8 pages.
[cited by applicant]
Arjovsky et al., “Towards principled methods for training generative adversarial network,” CoRR, Submitted on Jan. 2017, arXiv:1701.04862v1, 17 pages.
[cited by applicant]
Arjovsky et al., “Wasserstein generative adversarial networks,” International conference on machine learning, Jul. 2017, 10 pages.
[cited by applicant]
Atal et al., “Adaptive predictive coding of speech signals,” Bell System Technical Journal, Oct. 1970, 14 pages.
[cited by applicant]
Aytar et al., “Playing hard exploration games by watching youtube,” Advances in Neural Information Processing Systems, Dec. 2018, 12 pages.
[cited by applicant]
Ba et al., “Layer normalization,” CoRR, Submitted on Jul. 2016, arXiv:1607.06450v1, 14 pages.
[cited by applicant]
Bakker et al., “Robot see, robot do: An overview of robot imitation,” AISB96 Workshop on Learning in Robots and Animals, Apr. 1996, 10 pages.
[cited by applicant]
Baram et al., “End-to-end differentiable adversarial imitation learning,” Proceedings of the 34th International Conference on Machine Learning—vol. 70, Aug. 2017, 10 pages.
[cited by applicant]
Barth-Maron et al., “Distributed distributional deterministic policy gradients,” CoRR, Submitted on Apr. 2018, arXiv:1804.08617v1, 16 pages.
[cited by applicant]
Beattie et al., “Deepmind lab,” CoRR, Submitted on Dec. 2016, arXiv:1612.03801v1, 11 pages.
[cited by applicant]
Belghazi et al., “Mine: Mutual information neural estimation,” CoRR, Submitted on Jun. 2018, arXiv:1801.04062v4, 18 pages.
[cited by applicant]
Bellemare et al., “A distributional perspective on reinforcement learning,” CoRR, Submitted on Jul. 2017, arXiv:1707.06887v1, 19 pages.
[cited by applicant]
Bengio et al., “Adaptive importance sampling to accelerate training of a neural probabilistic language model,” IEEE Transactions on Neural Networks, Feb. 2008, 31 pages.
[cited by applicant]
Cho et al., “Learning phrase representations using rnn encoderdecoder for statistical machine translation,” CoRR, Submitted on Sep. 2014, arXiv:1406.1078v3, 15 pages.
[cited by applicant]
Chopra et al., “Learning a similarity metric discriminatively, with application to face verification,” CVPR, Jun. 2005, 8 pages.
[cited by applicant]
Clevert et al., “Fast and accurate deep network learning by exponential linear units (elus),” CoRR, Submitted on Feb. 2016, arXiv:1511.07289v5, 14 pages.
[cited by applicant]
Doersch et al., “Multi-task Self-Supervised Visual Learning,” Proceedings of the IEEE International Conference on Computer Vision, Aug. 2017, 13 pages.
[cited by applicant]
Doersch et al., “Unsupervised visual representation learning by context prediction,” Proceedings of the IEEE International Conference on Computer Vision, Dec. 2015, 9 pages.
[cited by applicant]
Donahue et al., “Adversarial feature learning,” CoRR, Submitted on Apr. 2017, arXiv:1605.09782v7, 18 pages.
[cited by applicant]
Duan et al., “One-shot imitation learning,” Advances in neural information processing systems, Dec. 2017, 12 pages.
[cited by applicant]
Elias, “Predictive coding—I,” IRE Transactions on Information Theory, Mar. 1955, 9 pages.
[cited by applicant]
Espeholt et al., “IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures,” CoRR, Submitted on Jun. 2018, arXiv:1802.01561v1, 22 pages.
[cited by applicant]
Finn et al., “A connection between generative adversarial networks, inverse reinforcement learning, and energy-based models,” CoRR, Submitted on Nov. 2016, arXiv:1611.03852v3, 10 pages.
[cited by applicant]
Finn et al., “One-shot visual imitation learning via meta-learning,” CoRR, Submitted on Sep. 2017, arXiv:1709.04905v1, 12 pages.
[cited by applicant]
Friston, “A theory of cortical responses,” Philosophical transactions of the Royal Society B: Biological sciences, Apr. 2005, 22 pages.
[cited by applicant]
Fu et al., “Learning robust rewards with adversarial inverse reinforcement learning,” CoRR, Submitted on Oct. 2017, arXiv:1710.11248v1, 15 pages.
[cited by applicant]
Goodfellow et al., “Generative adversarial nets,” Advances in neural information processing systems, Dec. 2014, 9 pages.
[cited by applicant]
Gutmann et al., “Noise-contrastive estimation: A new estimation principle for unnormalized statistical models,” Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Mar. 2010…
[cited by applicant]
He et al., “Identity mappings in deep residual networks,” CoRR, Submitted on Jul. 2016, arXiv:1603.05027v3, 15 pages.
[cited by applicant]
Hester et al., “Deep q-learning from demonstrations,” Thirty-Second AAAI Conference on Artificial Intelligence, Apr. 2018, 8 pages.
[cited by applicant]
Hinton et al., “Deep neural networks for acoustic modeling in speech recognition,” IEEE Signal processing magazine 29, Nov. 2012, 16 pages.
[cited by applicant]
Hinton et al., “Neural Networks for Machine Learning Lecture 6a Overview of mini--batch gradient descent,” UT Lectures from the course Neural Networks for Machine Learning , 2012, 31 pages.
[cited by applicant]
Ho et al., “Generative adversarial imitation learning,” Advances in neural information processing systems, Dec. 2016, 9 pages.
[cited by applicant]
Horgan et al., “Distributed prioritized experience replay,” CoRR, Submitted on Mar. 2018, arXiv1803.00933v1, 19 pages.
[cited by applicant]
Hosu et al., “Playing atari games with deep reinforcement learning and human checkpoint replay,” CoRR, Submitted on Jul. 2016, arXiv:1607.05077v1, 6 pages.
[cited by applicant]
Hu et al., “Mining and summarizing customer reviews,” Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, Apr. 2004, 10 pages.
[cited by applicant]
Hyvarinen et al., “Unsupervised feature extraction by time-contrastive learning and nonlinear ICA,” Advances in Neural Information Processing Systems, Dec. 2016, 9 pages.
[cited by applicant]
Ioffe et al., “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” CoRR, Submitted on Mar. 2015, arXiv:1502.03167v3, 11 pages.
[cited by applicant]
Jozefowicz et al., “Exploring the Limits of Language Modeling,” CoRR, Submitted on Feb. 2016, arXiv:1602.02410v2, 11 pages.
[cited by applicant]
Kingma et al., “Adam: A method for stochastic optimization,” CoRR, Submitted on Jan. 2017, arXiv:1412.6980v9, 15 pages.
[cited by applicant]
Kiros et al., “Skip-thought vectors,” Advances in neural information processing systems, Dec. 2015, 9 pages.
[cited by applicant]
Kostrikov et al., “Discriminator-Actor-Critic: Addressing sample inefficiency and reward bias in inverse reinforcement learning,” CoRR, Submitted on Oct. 2018, arXiv:1809.02925v2, 14 pages.
[cited by applicant]
Krizhevsky et al., “Imagenet classification with deep convolutional neural network,” Advances in neural information processing systems, Dec. 2012, 9 pages.
[cited by applicant]
Le et al., “Distributed representations of sentences and document,” International conference on machine learning, Jan. 2014, 9 pages.
[cited by applicant]
Li et al., “Infogail: Interpretable imitation learning from visual demonstrations,” Advances in Neural Information Processing Systems, Dec. 2017, 11 pages.
[cited by applicant]
Li et al., “Learning question classifiers,” Proceedings of the 19th international conference on Computational linguistics—vol. 1, Aug. 24-Sep. 1, 2002, 7 pages.
[cited by applicant]
Lillicrap et al., “Continuous control with deep reinforcement learning,” CoRR, Submitted on Sep. 2015, arXiv:1509.02971v1, 14 pages.
[cited by applicant]
Maaten et al., “Visualizing data using t-SNE,” Journal of machine learning research, Nov. 2008, 27 pages.
[cited by applicant]
Mikolov et al., “Efficient estimation of word representations in vector space,” CoRR, Submitted on Sep. 2013, arXiv:1301.3781v3 12 pages.
[cited by applicant]
Miyamoto et al., “A kendama learning robot based on bi-directional theory,” Neural Networks 9, Nov. 1996, 22 pages.
[cited by applicant]
Mnih et al., “A fast and simple algorithm for training neural probabilistic language models,” CoRR, Submitted on Jun. 2012, arXiv:1206.6426v1, 8 pages.
[cited by applicant]
Mnih et al., “Asynchronous methods for deep reinforcement learning,” International conference on machine learning, Jun. 2016, 10 pages.
[cited by applicant]
Ng et al., “Algorithms for inverse reinforcement learning,” Icml, Jun. 2000, 8 pages.
[cited by applicant]
Noroozi et al., “Unsupervised learning of visual representations by solving jigsaw puzzles,” CoRR, Submitted on Aug. 2017, arXiv:1603.09246v3, 19 pages.
[cited by applicant]
Oord et al., “Conditional image generation with pixelcnn decoders,” Advances in neural information processing systems, Dec. 2016, 9 pages.
[cited by applicant]
Oord et al., “Pixel recurrent neural networks,” CoRR, Submitted on Aug. 2016, arXiv:1601.06759v3, 11 pages.
[cited by applicant]
Oord et al., “Representation learning with contrastive predictive coding,” CoRR, Submitted on Jan. 2019, arXiv:1807.03748v2, 13 pages.
[cited by applicant]
Oord et al., “Wavenet: A generative model for raw audio,” CoRR, Submitted on Sep. 2016, arXiv:1609.03499v2, 15 pages.
[cited by applicant]
Panayotov et al., “Librispeech: an asr corpus based on public domain audio books,” 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr. 2015, 5 pages.
[cited by applicant]
Pang et al., “A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts,” CoRR, Submitted on Sep. 2004, arXiv:cs/0409058v1, 8 pages.
[cited by applicant]
Pang et al., “Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales,” CoRR, Submitted on Jun. 2005, arXiv:cs/0506075v1, 11 pages.
[cited by applicant]
Pohlen et al., “Observe and look further: Achieving consistent performance on atari,” CoRR, Submitted on May 2018, arXiv:1805.11593v1, 19 pages.
[cited by applicant]
Pomerleau et al., “Alvinn: An autonomous land vehicle in a neural network,” Advances in neural information processing systems, Jan. 1989, 9 pages.
[cited by applicant]
Povey et al., “The kaldi speech recognition toolkit,” IEEE 2011 workshop on automatic speech recognition and understanding, Dec. 2011, 4 pages.
[cited by applicant]
Radford et al., “Learning to Generate Reviews and Discovering Sentiment,” CoRR, Submitted on Apr. 2017, arXiv:1704.01444v2, 9 pages.
[cited by applicant]
Rao et al., “Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects,” Nature neuroscience, Jan. 1999, 9 pages.
[cited by applicant]
Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge,” CoRR, Submited on Jan. 2015, arXiv:1409.0575v3, 43 pages.
[cited by applicant]
Schaal et al., “Learning from demonstration,” Advances in neural information processing systems, 1997, 7 pages.
[cited by applicant]
Schaul et al., “Prioritized experience replay,” CoRR, Submitted on Feb. 2016, arXiv:1511.05952v4, 21 pages.
[cited by applicant]
Schroff et al., “FaceNet: A unified embedding for face recognition and clustering,” Proceedings of the IEEE conference on computer vision and pattern recognition, Jun. 2015, 9 pages.
[cited by applicant]
Sermanet et al., “Time-contrastive networks: Self-supervised learning from video,” CoRR, Submitted on Mar. 2018, arXiv:1704.06888v3, 15 pages.
[cited by applicant]
Silver et al., “Deterministic policy gradient algorithms, ” Proceedings of Machine Learning Research, Jun. 2014, 9 pages.
[cited by applicant]
Sutskever et al., “Sequence to sequence learning with neural networks,” Advances in neural information processing systems, Dec. 2014, 9 pages.
[cited by applicant]
Sutton et al., “Reinforcement learning: An introduction,” MIT press, Oct. 2018, 10 pages.
[cited by applicant]
Tassa et al., “Deepmind control suite,” CoRR, Submitted on Jan. 2018, arXiv:1801.00690v1, 24 pages.
[cited by applicant]
Ulyanov et al., “Instance normalization: the missing ingredient for fast stylization,” CoRR, Submitted on Nov. 2017, arXiv:1607.08022v3, 6 pages.
[cited by applicant]
Vaswani et al., “Attention is all you need,” Advances in neural information processing system, Dec. 2017, 11 pages.
[cited by applicant]
Vecerik et al., “Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards,” arXiv, 10 pages.
[cited by applicant]
Vinyals et al., “Show and tell: A neural image caption generator,” Proceedings of the IEEE conference on computer vision and pattern recognition, Dec. 2015, 9 pages.
[cited by applicant]
Wang et al., “Baselines and bigrams: Simple, good sentiment and topic classification,” Proceedings of the 50th annual meeting of the association for computational linguistics: Short papers—vol. 2, Jul. 2012, 5 pages.
[cited by applicant]
Wang et al., “Unsupervised Learning of Visual Representations using Videos,” CoRR, Submitted on Oct. 2015, arXiv:1505.00687v2, 9 pages.
[cited by applicant]
Weinberger et al., “Distance metric learning for large margin nearest neighbor classification,” Journal of Machine Learning Research 10, 38 pages.
[cited by applicant]
Wiebe et al., “Annotating expressions of opinions and emotions in language,” To appear in Language Resources and Evaluation 1, Jun. 2004, 54 pages.
[cited by applicant]
Wiskott et al., “Slow feature analysis: Unsupervised learning of invariances,” Neural computation, Apr. 2002, 56 pages.
[cited by applicant]
Zhang et al., “Colorful image colorization,” CoRR, Submitted on Oct. 2016, arXiv:1603.08511v5, 29 pages.
[cited by applicant]
Zhao et al., “Self-adaptive hierarchical sentence model,” Twenty-Fourth International Joint Conference on Artificial Intelligence, Jun. 2015, 8 pages.
[cited by applicant]
Zhu et al., “Aligning books and movies: Towards story-like visual explanations by watching movies and reading books,” Proceedings of the IEEE international conference on computer vision, Dec. 2015, 9 pages.
[cited by applicant]
Zhu et al., “Reinforcement and imitation learning for diverse visuomotor skills,” CoRR, Submitted on May 2018, arXiv:1802.09564v2, 12 pages.
[cited by applicant]
Ziebart et al., “Maximum entropy inverse reinforcement learning,” AAAI, Jul. 2008, 6 pages.
[cited by applicant]