IP Library › Granted Patent US 12,528,186
Granted Patent B2
US 12,528,186 · App. 17/924,891 · Granted Jan 20, 2026

Training and/or utilizing machine learning model(s) for use in natural language based robotic control

Inventors: Pierre Sermanet (Palo Alto, CA); Corey Lynch (Mountain View, CA)
Assignee: GOOGLE LLC
B25J9/1664B25J9/163B25J9/1697
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,528,186
App. No.
17/924,891
Granted
Jan 20, 2026
Kind
B2
Abstract

Techniques are disclosed that enable training a goal-conditioned policy based on multiple data sets, where each of the data sets describes a robot task in a different way. For example, the multiple data sets can include: a goal image data set, where the task is captured in the goal image; a natural language instruction data set, where the task is described in the natural language instruction; a task ID data set, where the task is described by the task ID, etc. In various implementations, each of the multiple data sets has a corresponding encoder, where the encoders are trained to generate a shared latent space representation of the corresponding task description. Additional or alternative techniques are disclosed that enable control of a robot using a goal-conditioned policy network. For example, the robot can be controlled, using the goal-conditioned policy network, based on free-form natural language input describing robot task(s).

Claims (60)

1 . A method implemented by one or more processors, the method comprising:

receiving a free-form natural language instruction describing a task for a robot, the free-form natural language instruction generated based on user interface input provided by a user via one or more user interface input devices;

processing the free-form natural language instruction using a natural language instruction encoder, to generate a latent goal representation of the free-form natural language instruction, wherein the latent goal representation of the free-form natural language instruction is represented in a shared latent space with latent goal representations of goal images;

receiving an instance of vision data, the instance of vision data generated by at least one vision component of the robot, and the instance of vision data capturing at least part of an environment of the robot;

generating output based on processing, using a goal-conditioned policy network, at least (a) the instance of vision data and (b) the latent goal representation of the free-form natural language instruction,

wherein the goal-conditioned policy network is trained based on at least:

(i) a goal image set of training instances, in which training tasks are described using goal images, wherein one or more of the goal images comprise a last image frame of a sequence of image frames selected from a larger data stream of image frames that capture states of one or more robots and corresponding actions of the one or more robots interacting with one or more environments; and

(ii) a natural language instruction set of training instances, in which training tasks are described using free-form natural language instructions; and

controlling one or more actuators of the robot based on the generated output, wherein controlling the one or more actuators of the robot causes the robot to perform at least one action indicated by the generated output.

2 . The method of claim 1 , further comprising:

receiving an additional free-form natural language instruction describing an additional task for the robot, the additional free-form natural language instruction generated based on additional user interface input provided by the user via the one or more user interface input devices;

processing the additional free-form natural language instruction using the natural language instruction encoder, to generate an additional latent goal representation of the additional free-form natural language instruction;

receiving an additional instance of vision data generated by the at least one vision component of the robot;

generating, using the goal-conditioned policy network, additional output based on processing at least (a) the additional instance of vision data and (b) the additional latent goal representation of the additional free-form natural language instruction; and

controlling the one or more actuators of the robot based on the generated additional output, wherein controlling the one or more actuators of the robot causes the robot to perform at least one additional action indicated by the generated additional output.

3 . The method of claim 2 , wherein the additional task for the robot is distinct from the task for the robot.

4 . The method of claim 3 , wherein each training instance in the goal image set of training instances, in which training tasks are described using goal images, comprises an imitation trajectory provided by a human and a goal image describing the training task performed by robot in the imitation trajectory.

5 . The method of claim 1 , wherein each training instance, in the natural language instruction set of training instances, in which training are described using free-form natural language instructions, comprises an imitation trajectory provided by a human and a free-form natural language instruction describing the training task performed by robot in the imitation trajectory.

6 . The method of claim 5 , wherein generating each training instance is the natural language instruction set of training instances comprises:

receiving a data stream, capturing the state of the robot and corresponding actions of the robot, while the human is controlling the robot to interact with the environment;

for each training instance in the natural language instruction set of training instances:

selecting a sequence of image frames from the data stream;

providing the sequence of image frames to a human reviewer;

receiving a training free-form natural language instruction describing a training task performed by the robot in the sequence of image frames;

generating the training instance by storing, as the training instance, the selected sequence of image frames as the imitation trajectory portion of the training instance, and the training free-form natural language instruction as the free-form natural language instruction portion of the training instance.

7 . The method of claim 6 , wherein training the goal-conditioned policy network, based on at least (i) the goal image set of training instances, in which training tasks are described using goal images, and (ii) the natural language instruction set of training instances, in which training tasks are described using free-form natural language instructions, comprises:

selecting a first training instance from the goal image set of training instances, wherein the first training instance includes a first imitation trajectory and a first goal image describing the first imitation trajectory;

generating a latent space representation of the first goal image by processing, using a goal image encoder, the first goal image portion of the first training instance;

processing, using the goal-conditioned policy network, at least (1) the initial image frame in the first imitation trajectory and (2) the latent space representation of the first goal image portion of the first training instance, to generate first candidate output;

determining a goal image loss based on the first candidate output and one or more portions of the first imitation trajectory;

selecting a second training instance from the natural language instruction set of training instances, wherein the second training instance includes a second imitation trajectory and a second free-form natural language instruction describing the second imitation trajectory;

generating a latent space representation of the second free-form natural language instruction by processing, using the natural language encoder, the second free-form natural language instruction portion of the second training instance, wherein the latent space representation of the first goal image and the latent space representation of the second free-form natural language instruction are represented in a shared latent space;

processing, using the goal-conditioned policy network, at least (1) the initial image frame in the second imitation trajectory and (2) the latent space representation of the second free-form natural language instruction portion of the second training instance, to generate second candidate output;

determining a natural language instruction loss based on the second candidate output and one or more portions of the second imitation trajectory;

determining a goal-conditioned loss based on the goal image loss and the natural language instruction loss; and

updating one or more portions of the goal image encoder, the natural language instruction encoder, and/or the goal-conditioned policy network, based on the determined goal-conditioned loss.

8 . The method of claim 7 , wherein the goal-conditioned policy network is trained, based on a first quantity of training instances of the goal image set of training instances, and a second quantity of training instances of the natural language instruction set of training instances, wherein the second quantity is less than fifty percent of the first quantity.

9 . The method of claim 8 , wherein the second quantity is less than ten percent of the first quantity, less than five percent of the first quantity, or less than one percent of the first quantity.

10 . The method of claim 9 , wherein the generated output comprises a probability distribution over an action space of the robot, and wherein controlling the one or more actuators based on the generated output comprises selecting the at least one action based on the at least one action with the highest probability in the probability distribution.

11 . The method of claim 9 , wherein the generating output based on processing, using the goal-conditioned policy network, at least (a) the instance of vision data and (b) the latent goal representation of the free-form natural language instruction further comprises generating output based on processing, using the goal-conditioned policy network, (c) the at least one action, and wherein controlling the one or more actuators based on the generated output comprises selecting the at least one action based on the at least one action satisfying a threshold probability.

12 . A method implemented by one or more processors, the method comprising:

selecting a first training instance from the goal image set of training instances, wherein the first training instance includes a first imitation trajectory and a first goal image describing the first imitation trajectory;

generating a latent space representation of the first goal image by processing, using a goal image encoder, the first goal image portion of the first training instance;

processing, using a goal-conditioned policy network, at least (1) the initial image frame in the first imitation trajectory and (2) the latent space representation of the first goal image portion of the first training instance, to generate first candidate output;

determining a goal image loss based on the first candidate output and one or more portions of the first imitation trajectory;

selecting a second training instance from the natural language instruction set of training instances, wherein the second training instance includes a second imitation trajectory and a second free-form natural language instruction describing the second imitation trajectory;

generating a latent space representation of the second free-form natural language instruction by processing, using the natural language encoder, the second free-form natural language instruction portion of the second training instance, wherein the latent space representation of the first goal image and the latent space representation of the second free-form natural language instruction are represented in a shared latent space;

processing, using the goal-conditioned policy network, at least (1) the initial image frame in the second imitation trajectory and (2) the latent space representation of the second free-form natural language instruction portion of the second training instance, to generate second candidate output;

determining a natural language instruction loss based on the second candidate output and one or more portions of the second imitation trajectory;

determining a goal-conditioned loss based on the goal image loss and the natural language instruction loss; and

updating one or more portions of the goal image encoder, the natural language instruction encoder, and/or the goal-conditioned policy network, based on the determined goal-conditioned loss.

13 . A non-transitory computer-readable storage medium storing instructions executable by one or more processors of a computing system to perform the method of:

receiving a free-form natural language instruction describing a task for a robot, the free-form natural language instruction generated based on user interface input provided by a user via one or more user interface input devices;

processing the free-form natural language instruction using a natural language instruction encoder, to generate a latent goal representation of the free-form natural language instruction, wherein the latent goal representation of the free-form natural language instruction is represented in a shared latent space with latent goal representations of goal images;

receiving an instance of vision data, the instance of vision data generated by at least one vision component of the robot, and the instance of vision data capturing at least part of an environment of the robot;

generating output based on processing, using a goal-conditioned policy network, at least (a) the instance of vision data and (b) the latent goal representation of the free-form natural language instruction,

wherein the goal-conditioned policy network is trained based on at least:

(i) a goal image set of training instances, in which training tasks are described using goal images, wherein one or more of the goal images comprise a last image frame of a sequence of image frames selected from a larger data stream of image frames that capture states of one or more robots and corresponding actions of the one or more robots interacting with one or more environments; and

(ii) a natural language instruction set of training instances, in which training tasks are described using free-form natural language instructions; and

controlling one or more actuators of the robot based on the generated output, wherein controlling the one or more actuators of the robot causes the robot to perform at least one action indicated by the generated output.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2022
From: SERMANET, PIERRE; LYNCH, COREY
To: GOOGLE LLC
Reel/Frame 061754/0732 →
Continuity (2)
Provisional Application 63024996 · May 14, 2020
Related Publication 20230182296A1 · Jun 15, 2023
References Cited (103)
US 20190206400A1 · Cui · 2019 [cited by applicant]
US 20200103978A1 · Mixter et al. · 2020 [cited by applicant]
US 20200104680A1 · Reed · 2020 [cited by applicant]
KR 20180137026 · 2018 [cited by applicant]
KR 101997072 · 2019 [cited by applicant]
WO 2019183568 · 2019 [cited by applicant]
Luke Richards et al., ‘A Manifold Alignment Approach to Grounded Language Learning’, 2019, 8th Northeast Robotics Colloquium. (Year: 2019). [cited by examiner]
Rahmatizadeh, Rouhollah et al.; Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration; IEEE International Conference on Robotics and Automation; 8 pages; dated Apr. 201… [cited by applicant]
Kalashnikov, Dmitry, et al. “QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation.” arXiv:1806.10293v3 [cs.LG] Nov. 28, 2018; 23 pages. [cited by applicant]
Sermanet, P. et al., “Time-Contrastive Networks: Self-Supervised Learning from Video;” 2018 International Conference on Robotics and Automation; pp. 1134-1141; May 21, 2018. [cited by applicant]
Rajeswaran, Aravind et al.; Learning complex dexterous manipulation with deep reinforcement learning and demonstrations; arXiv preprint arXiv:1709.10087; 9 pages; dated 2017. [cited by applicant]
Schaul, Tom et al.; Universal value function approximators; In International Conference on Machine Learning; 9 pages; dated 2015. [cited by applicant]
Todorov, Emanuel et al.; Mujoco: A physics engine for model-based control. In Intelligent Robots and Systems (IROS), IEEE/RSJ; p. 5026-5033; dated 2012. [cited by applicant]
Levine, S. et al. “End-to-End Training of Deep Visuomotor Policies;” Journal of Machine Learning Research, 17; 40 pages; Apr. 2016. [cited by applicant]
Radford, A.; Language Models are Unsupervised Multitask Learners; 24 pages; dated 2019. [cited by applicant]
Haarnoja, T. et al., “Soft Actor-Critic Algorithms and Applications;” Cornell University; arXiv.org; arXiv:1812.05905v1; 18 pages; Dec. 13, 2018. [cited by applicant]
Van den Oord, A. et al., “Representation Learning with Contrastive Predictive Coding,” Cornell University; arXiv.org; arXiv:1807.03748v1; 13 pages; Jul. 10, 2018. [cited by applicant]
Huang, H. et al., “Transferable Representation Learning in Vision-and-Language Navigation;” IEEE/CVF International Conference on Computer Vision (ICCV); pp. 7404-7413; Oct. 27, 2019. [cited by applicant]
Chai, J. et al., “Language to Action: Towards Interactive Task Learning with Physical Agents;” Proceedings of the 27th International Joint Conference on Artificial Intelligence; 8 pages; Jul. 1, 2018. [cited by applicant]
Lynch, C. et al., “Learning Latent Plans from Play;” arXiv.org, arXiv:1903.01973v1; 14 pages; dated Mar. 5, 2019. [cited by applicant]
Stepputtis, S. et al., “Imitation Learning of Robot Policies by Combining Language, Vision and Demonstration;” arXiv.org, arXiv:1911.11744v1; 6 pages; dated Nov. 26, 2019. [cited by applicant]
Sharma, P. et al., “Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation;” arXiv.org, arXiv: 1810.07121v1; 10 pages; dated Oct. 16, 2018. [cited by applicant]
Wang, X. et al., “Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation;” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 10 pages; Jun. 15, 20… [cited by applicant]
European Patent Office; International Search Report and Written Opinion of PCT/US2021/032499; 16 pages; dated Aug. 26, 2021. [cited by applicant]
Atkeson, C. G., et al., “Robot Learning from Demonstration;” In ICML, vol. 97; 9 pages; dated 1997. [cited by applicant]
Mooney, R.J., “Learning to Connect Language and Perception;” In Association for the Advancement of Artificial Intelligence (AAAI); pp. 1598-1601; dated 2008. [cited by applicant]
Oh, J. et al., “Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning;” In Proceedings of the 34th International Conference on Machine Learning—vol. 70, 10 pages; dated 2017. [cited by applicant]
Raffel, C. et al., “Exploring the Limits of Transfer Learning with a Unified To-Text Transformer;” arXiv.org; arXiv:1910.10683v2; 53 pages; dated Oct. 24, 2019. [cited by applicant]
Salimans, T. et al., “PixelCNN++: Improving the pixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications;” International Conference on Learning Representations; 10 pages; dated 2017. [cited by applicant]
Schaul, T. et al., “Ray Interference: A Source of Plateaus in Deep Reinforcement Learning;” arXiv.org; arXiv:1904.11455v1; 17 pages; dated Apr. 25, 2019. [cited by applicant]
Schmidhuber, Jurgen, “Formal Theory of Creativity, Fun, and Intrinsic Motivation;” IEEE Transactions on Autonomous Mental Development, vol. 2, No. 3; pp. 230-247; dated Sep. 2010. [cited by applicant]
Sennrich, R. et al., “Neural Machine Translation of Rare Words with Subword Units;” arXiv.org; arXiv:1508.07909v2; 11 pages; dated Nov. 27, 2015. [cited by applicant]
Sermanet, P. et al., “Unsupervised Perceptual Rewards for Imitation Learning;” Proceedings of Robotics: Science and Systems (RSS), 2017; arxiv.org; arXiv:1612.06699v3; 15 pages; dated 2017. [cited by applicant]
Shridhar, M. et al., “Alfred: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks;” In the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020; arxiv.org; arXiv:1912.01734v2; 18 page… [cited by applicant]
Singh, A. et al., “End-to-End Robotic Reinforcement Learning Without Reward Engineering;” arXiv.org; arXiv:1904.07854v2; 14 pages; dated Mar. 16, 2019. [cited by applicant]
Singh, A. et al., “Scalable Multi-Task Imitation Learning with Autonomous Improvement;” arXiv.org; arXiv:2003.02636v1; 7 pages; dated Feb. 25, 2020. [cited by applicant]
Smith, L. et al., “The Development of Embodied Cognition: Six Lessons from Babies;” Artificial Life, vol. 11, Issue 1-2; pp. 13-29; dated Jan. 2005. [cited by applicant]
Tan, C. et al., “A Survey on Deep Transfer Learning;” In International Conference on Artificial Neural Networks; Springer; 10 pages, dated 2018. [cited by applicant]
Teh, Y.W. et al., “Distral: Robust Multitask Reinforcement Learning;” In Advances in Neural Information Processing Systems; 11 pages; dated 2017. [cited by applicant]
Tenney, I. et al., “What do you learn from context? probing for sentence structure in contextualized word representations;” International Conference on Learning Representations (ICLR); 17 pages; dated 2019. [cited by applicant]
Thrun, S.B.; “Efficient Exploration in Reinforcement Learning;” Citeseer; Technical Report CMU-CS-92-102; 44 pages; dated 1992. [cited by applicant]
Verga, L. et al., “How Relevant Is Social Interaction in Second Language Learning?” Frontiers in Human Neuroscience, vol. 7, Article 550; 7 pages; dated Sep. 2013. [cited by applicant]
Warde-Farley, D. et al., “Unsupervised Control Through Non-Parametric Discriminative Rewards;” arXiv.org; arXiv:1811.11359v1; 17 pages; dated Nov. 28, 2018. [cited by applicant]
Yang, Y. et al., “Multilingual Universal Sentence En-coder for Semantic Retrieval;” arXiv.org; arXiv:1907.04307v1; 6 pages; dated Jul. 9, 2019. [cited by applicant]
Yu, H. et al., “A Deep Compositional Framework for Human-Like Language Acquisition in Virtual Environment;” arXiv.org; arXiv:1703.09831v3; 16 pages; dated May 19, 2017. [cited by applicant]
Yu, H. et al., “Interactive Grounded Language Acquisition and Generalization in a 2D World;” arXiv.org; arXiv:1802.01433v4; 29 pages; dated Aug. 13, 2013. [cited by applicant]
Yu, T. et al., “Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning;” arXiv.org; arXiv:1910.10897v1; 18 pages; dated Oct. 24, 2019. [cited by applicant]
Zellers, R. et al., “Swag: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference;” arXiv.org; arXiv:1808.05326v1; 15 pages; dated Aug. 16, 2018. [cited by applicant]
Zhang, T. et al., “Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation;” In IEEE International Conference on Robotics and Automation (ICRA); pp. 5628-5635; dated 2018. [cited by applicant]
Japanese Patent Office; Notice of Reasons for Rejection issued in Application No. 2022565890; 6 pages; dated Sep. 4, 2023. [cited by applicant]
European Patent Office; Communication pursuant to Article 94(3) EPC issued in Application No. 21730747.9 9 pages; dated Aug. 13, 2024. [cited by applicant]
Andreas, J. et al., “Modular Multitask Reinforcement Learning with Policy Sketches;” International Conference on Machine Learning (ICML); 10 pages; dated 2016. [cited by applicant]
Andrychowicz, M. et al.; Hindsight Experience Replay;31st Conference on Neural Information Processing Systems; 11 pages; dated 2017. [cited by applicant]
Andrychowicz, M. et al., Open AI; “Learning Dexterous In-Hand Manipulation;” The International Journal of Robotics Research, 39(1); pp. 3-20; dated 2020. [cited by applicant]
Argall, B. et al. “A Survey of Robot Learning from Demonstration”; Robotics and Autonomous Systems, Elsevier BV, vol. 57, No. 5, pp. 469-483; May 31, 2009. [cited by applicant]
Bisk, Y. et al., “Experience Grounds Language;” arXiv.org; arXiv:2004.10151v3; 18 pages; dated Nov. 2, 2020. [cited by applicant]
Caruana, R.; “Multitask Learning;” Springer; Machine learning, 28(1) pp. 41-75; dated 1997. [cited by applicant]
Chaplot, D.S. et al., “Gated-Attention Architectures for Task-Oriented Language Grounding;” In Thirty-Second AAAI Conference on Artificial Intelligence; 8 pages; dated 2018. [cited by applicant]
Clark, H.H. et al., “Grounding in Communication;” American Psychological Association; psycnet.apa.org; 12 pages; dated 1991. [cited by applicant]
Das, A. et al., “Embodied Question Answering;” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR); 10 pages; dated 2018. [cited by applicant]
Devlin, J. et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding;” arXiv preprint arXiv:1810.04805v1; 14 pages; dated Oct. 11, 2018. [cited by applicant]
Ding, Y. et al., “Goal-conditioned Imitation Learning;” In Advances in Neural Information Processing Systems; 12 pages; dated 2019. [cited by applicant]
Duan, Y. et al., “One-Shot Imitation Learning;” In Advances in Neural Information Processing Systems; 12 pages, dated 2017. [cited by applicant]
Ebert, F. et al., Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control; arXiv.org; arXiv:1812.00568v1; 14 pages; dated Dec. 3, 2018. [cited by applicant]
Eysenbach, B. et al., Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement; arXiv.org; arXiv:2002.11089v1; 16 pages; dated Feb. 25, 2020. [cited by applicant]
Florensa, C. et al. “Self-supervised Learning of Image Embedding for Continuous Control;” arXiv.org; arXiv:1901.00943v1; 11 pages; dated Jan. 3, 2019. [cited by applicant]
Ghosh, D. et al., “Learning to Reach Goals without Reinforcement Learning;” arXiv.org; arXiv:1912.06088v2; 16 pages; dated Dec. 13, 2019. [cited by applicant]
Goldberg, Y. et al., “Assessing BERT's Syntactic Abilities;” arXiv.org; arXiv:1901.05287v1; 4 pages; dated Jan. 16, 2019. [cited by applicant]
Goyal, P. et al., Using Natural Language for Reward Shaping in Reinforcement Learning; arXiv.org; arXiv:1903.02020v2; 10 pages; dated May 31, 2019. [cited by applicant]
Gu, S. et al., “Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates;” IEEE International Conference on Robotics and Automation (ICRA); pp. 3389-3396; dated 2017. [cited by applicant]
Gupta, A. et al., “Relay Policy Learning: Solving Long Horizon Tasks via Imitation and Reinforcement Learning;” Conference on Robot Learning (CoRL); 13 pages; dated 2019. [cited by applicant]
Ha, D. et al., “World Models;” arXiv.org; arXiv:1803.10122v4; 21 pages; dated May 9, 2018. [cited by applicant]
Hafner, D. et al., “Learning Latent Dynamics for Planning from Pixels;” arXiv.org; arXiv:1811.04551v2; 18 pages; dated Dec. 3, 2018. [cited by applicant]
Handa, A. et al., “DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System;” arXiv.org; arXiv:1910.03135v2; 18 pages; dated Oct. 14, 2019. [cited by applicant]
Harnad, S., “The Symbol Grounding Problem;” Physica D: Nonlinear Phenomena, 42(1-3); 15 pages; dated 1990. [cited by applicant]
Hermann, K. M. et al., “Grounded Language Learning in a Simulated 3D World;” arXiv.org; arXiv:1706.06551v2; 22 pages; dated Jun. 26, 2017. [cited by applicant]
Hester, T. et al., “Deep Q-Learning from Demonstrations;” The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), 8 pages; 2018. [cited by applicant]
Hill, F. et al., “Emergent Systematic Generalization in a Situated Agent;” arXiv preprint arXiv:1910.00571v2; 14 pages; dated Oct. 28, 2019. [cited by applicant]
Howard, J. et al., “Universal Language Model Fine-turning for Text Classification;” arXiv.org; arXiv:1801.06146v5; 12 pages; dated May 23, 2018. [cited by applicant]
Jiang et al., “Language as an Abstraction for Hierarchical Deep Reinforcement Learning” arXiv:1906.07343v2 [cs.LG] dated Nov. 18, 2019. 25 pages. [cited by applicant]
Kaelbling, L.P., “Learning to Achieve Goals;” In IJCAI; Citeseer; 5 pages; dated 1993. [cited by applicant]
Kober, J. et al., “Reinforcement Learning in Robotics: A Survey;” The International Journal of Robotics Research; 37 pages; 2013. [cited by applicant]
Kollar, T. et al.; “Toward Understanding Natural Language Directions;” In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI); pp. 259-266; dated 2010. [cited by applicant]
Kolter, J. Z. et al., “Near-Bayesian Exploration in Polynomial Time;” In Proceedings of the 26th Annual Interna-tional Conference on Machine Learning; 10 pages; dated 2009. [cited by applicant]
Lee, L. et al., “Efficient Exploration via State Marginal Matching;” arXiv.org; arXiv:1906.05274v2; 19 pages; dated Oct. 4, 2019. [cited by applicant]
Luketina, J. et al., “A Survey of Reinforcement Learning Informed by Natural Language;” arXiv.org; arXiv:1906.03926v1; 9 pages; dated Jun. 10, 2019. [cited by applicant]
Lynch, C. et al., “Learning Latent Plans from Play;” Conference on Robot Learning (CoRL); https://arxiv.org/abs/1903.01973v2; 17 pages; dated Dec. 20, 2019. [cited by applicant]
MacMahon, M. et al., “Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions;” Def, 2(6):4; 8 pages; dated 2006. [cited by applicant]
Misra, D. et al., Mapping Instructions and Visual Observations to Actions with Reinforcement Learning; arXiv.org; arXiv:1704.08795v2; 16 pages; dated Jul. 22, 2017. [cited by applicant]
Nair, A. et al., “Contextual Imagined Goals for Self-Supervised Robotic Learning;” arXiv.org; arXiv:1910.11670v1; 12 pages; dated Oct. 23, 2019. [cited by applicant]
Nair, A. et al., “Visual Reinforcement Learning with Imagined Goals;” In Advances in Neural Information Processing Systems; 10 pages; dated 2018. [cited by applicant]
Oh, J. et al.; Action-conditional Video Prediction Using Deep Networks in Atari Games. In Advances in Neural Information Processing Systems; pp. 1-9; dated 2015. [cited by applicant]
Oudeyer, P-Y. et al., “How Can We Define Intrinsic Motivation?” In Proceedings of the 8th Conference on Epigenetic Robotics; 9 pages; dated 2008. [cited by applicant]
Ozair, S. et al., “Wasserstein Dependency Measure for Representation Learning;” In Advances in Neural Information Processing Systems; 11 pages; dated 2019. [cited by applicant]
Parisotto, E. et al., “Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning;” arXiv.org; arXiv:1511.06342v2; 15 pages; dated Nov. 20, 2015. [cited by applicant]
Pathak, D. et al., “Curiosity-driven Exploration by Self-supervised Prediction;” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops; pp. 16-17; dated 2017. [cited by applicant]
Pirk, S. et al., “Online Object Representations with Contrastive Learning;” arXiv.org; arXiv:1906.04312v1; 15 pages; dated Jun. 10, 2019. [cited by applicant]
Pong, V. et al., “Temporal Difference Models: Model-Free Deep RL for Model-Based Control;” arXiv.org; arXiv:1802.09081v1; 14 pages; dated Feb. 25, 2018. [cited by applicant]
Pong, V. H. et al., “Skew-Fit: State-Covering Self-Supervised Reinforcement Learning;” arXiv.org; arXiv:1903.03698v2; 23 pages; dated May 31, 2019. [cited by applicant]
Popov, I. et al., “Data-efficient Deep Reinforcement Learning for Dexterous Manipulation;” arXiv.org; arXiv:1704.03073; 12 pages; dated 2017. [cited by applicant]
China National Intellectual Property Administration; Notification of First Office Action issued in Application No. 202180034023.2; 17 pages; dated Jan. 14, 2025. [cited by applicant]
China National Intellectual Property Administration; Notice of Grant issued in Application No. 202180034023.2; 6 pages; dated May 21, 2025. [cited by applicant]
Korean Patent Office; Notice of Office Action issued in Application No. 1020227042611; 14 pages; dated Dec. 18, 2024. [cited by applicant]