IP Library › Granted Patent US 12,533,800
Granted Patent B2
US 12,533,800 · App. 17/103,827 · Granted Jan 27, 2026

Training reinforcement learning agents to learn farsighted behaviors by predicting in latent space

Inventors: Danijar Hafner (London, GB); Mohammad Norouzi (Richmond Hil, CA); Timothy Paul Lillicrap (London, GB)
Assignee: Google LLC
B25J9/163G06F18/211G06F18/213G06F30/27G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,533,800
App. No.
17/103,827
Granted
Jan 27, 2026
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection policy neural network used to select an action to be performed by an agent interacting with an environment. In one aspect, a method includes: receiving a latent representation characterizing a current state of the environment; generating a trajectory of latent representations that starts with the received latent representation; for each latent representation in the trajectory: determining a predicted reward; and processing the state latent representation using a value neural network to generate a predicted state value; determining a corresponding target state value for each latent representation in the trajectory; determining, based on the target state values, an update to the current values of the policy neural network parameters; and determining an update to the current values of the value neural network parameters.

Claims (87)

1 . A method of training a policy neural network having a plurality of policy neural network parameters used to select actions to be performed by an agent to control the agent in an environment, the method comprising:

obtaining an initial observation that characterizes an initial state of the environment;

processing at least the initial observation by using a representation neural network having a plurality of representation neural network parameters to generate an initial latent representation that characterizes the initial state of the environment, wherein the initial latent representation is in a latent space;

generating a trajectory of latent representations that comprises a sequence of multiple latent representations that are in the latent space and that each characterize a different state of the environment, wherein the sequence of multiple latent representations start with the initial latent representation that characterizes the initial state of the environment, and wherein generating the trajectory of latent representations comprises, at each subsequent step that is after a beginning step in a plurality of steps:

processing a current latent representation that is generated by using a transition neural network and without processing any observation that characterizes any state of the environment using the policy neural network and in accordance with current values of the plurality of policy neural network parameters to generate an action selection output;

selecting, using the action selection output, an action that would be performed by the agent when the environment is in a current state characterized by the current latent representation; and

processing the current latent representation that is generated by using the transition neural network and the action that is selected using the policy neural network and without processing any observation that characterizes any state of the environment using the transition neural network having a plurality of transition neural network parameters to generate an updated latent representation that is in the latent space and that characterizes a next state that the environment would transition into if the agent performed the selected action when the environment is in the current state characterized by the current latent representation;

for each latent representation in the sequence of multiple latent representations included in the trajectory that are in the latent space:

determining a predicted reward; and

processing the latent representation and without processing any observation that characterizes any state of the environment using a value neural network having a plurality of value neural network parameters and in accordance with current values of the plurality of value neural network parameters to generate a predicted state value that represents a time-adjusted sum of predicted rewards that would be received by the agent by selecting actions using action selection outputs starting from a state characterized by the latent representation;

determining a target state value for each latent representation in the trajectory from the predicted rewards and the predicted state values for the latent representations in the trajectory;

training, based on the predicted state values that have been generated by using the value neural network from the latent representations in the latent space, the policy neural network to update the current values of the plurality of policy neural network parameters, wherein training the policy neural network comprises determining an update to the current values of the plurality of policy neural network parameters that encourages the policy neural network to generate action selection outputs that result in actions being selected that improve the predicted state values; and

training, based on the predicted state values that have been generated by using the value neural network from the latent representations in the latent space, the value neural network to update the current values of the plurality of value neural network parameters, wherein training the value neural network comprises determining an update to the current values of the plurality of value neural network parameters by optimizing a value objective function that measures a difference between the predicted state values and the target state values.

2 . The method of claim 1 , wherein determining the predicted reward comprises:

processing the latent representation using a reward neural network having a plurality of reward neural network parameters.

3 . The method of claim 1 , further comprising:

collecting experiences generated as a result of the agent interacting with the environment, wherein each experience comprises (i) a respective current observation characterizing a respective current state of the environment, (ii) a respective current action performed by the agent in response to the current observation, and (iii) a respective current reward received by the agent in response to the agent performing the respective current action.

4 . The method of claim 3 , further comprising:

generating a plurality of training tuples based on the experiences, each training tuple comprising (i) a respective current observation characterizing a respective current state of the environment, (ii) a respective preceding action performed by the agent, (iii) a respective preceding latent representation characterizing a respective preceding state of the environment;

for each training tuple:

generating, by processing the training tuple using the representation neural network, a current latent representation that characterizes a current state of the environment; and

determining, based on computing a gradient of a representation learning objective function with respect to the representation neural network parameters, an update to current values of the representation neural network parameters.

5 . The method of claim 4 , further comprising, when the training tuple is a first training tuple in the plurality of training tuples:

generating, by processing a current observation that is specified by the training tuple using the representation neural network, a current latent representation that characterizes an initial state of the environment.

6 . The method of claim 4 , wherein the representation learning objective function evaluates respective qualities of at least one of reward prediction, observation reconstruction, or contrastive estimation.

7 . The method of claim 4 , further comprising, for each training tuple:

generating, by processing the current latent representation that is generated by the representation neural network using the reward neural network, a current reward to be received by the agent in response to the agent performing the preceding action; and

generating, by processing the preceding action performed by the agent and the preceding latent representation that are specified by the training tuple using the transition neural network, an estimated current latent representation emulating the current latent representation generated by using the representation neural network and that characterizes the current state of the environment.

8 . The method of claim 7 , wherein determining the update to current values of the representation neural network further comprises:

determining, based on computing respective gradients of the representation learning objective function with respect to the reward neural network parameters and the transition neural network parameters, respective updates to current values of the reward neural network parameters and the transition neural network parameters.

9 . The method of claim 1 , further comprising, performing the following at each of a plurality of execution steps:

generating, by processing a respective training tuple using the representation neural network, a current latent representation that characterizes the current state of the environment;

generating, by processing the generated latent representation using the policy neural network, a current action selection output; and

controlling the agent to perform an action that is selected using the current action selection output, the action causing the environment to transition into a subsequent state.

10 . The method of claim 9 , wherein generating the action selection output further comprises:

adding exploration noise to the action selection output.

11 . The method of claim 9 , wherein controlling the agent to perform the action that is selected using the current action selection output further comprises:

receiving, from the environment and by the agent, a reward in response to the agent performing the current action; and

obtaining an updated observation characterizing the subsequent state of the environment.

12 . The method of claim 11 , further comprising:

generating a new experience by associating a current observation that is specified by the training tuple with the current action performed by the agent and with the reward received by the agent.

13 . The method of claim 1 , wherein determining the corresponding target state value for each latent representation in the trajectory comprises:

determining a respective candidate target state value for each of one or more values of K, comprising, for each of the values of K:

determining a time-adjusted sum of the predicted rewards over H latent representations starting from a current latent representation, and

determining an estimated state value for a latent representation that is H latent representations after the current latent representation,

wherein the respective H for any given latent representation equals a minimum of (i) a sum between a numeric position of the given latent representation in the trajectory and K, and (ii) a total number of latent representations in the trajectory, and wherein each value of K is a different positive integer.

14 . The method of claim 1 , wherein determining the corresponding target state value for each latent representation in the trajectory further comprises:

computing an exponentially-weighted average of the respective candidate target state values for the one or more values of K.

15 . The method of claim 1 , wherein training the policy neural network comprises computing gradients of the target state values with respect to the plurality of policy neural network parameters through reparameterized sampling techniques.

16 . The method of claim 1 , wherein training the policy neural network comprises computing gradients of the target state values with respect to the plurality of policy neural network parameters through straight-through gradient estimation techniques.

17 . The method of claim 3 , wherein collecting experiences generated as a result of the agent interacting with the environment comprises controlling the agent to perform random actions in the environment.

18 . A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations for training a policy neural network having a plurality of policy neural network parameters used to select actions to be performed by an agent to control the agent in an environment, the operations comprising:

obtaining an initial observation that characterizes an initial state of the environment;

processing at least the initial observation by using a representation neural network having a plurality of representation neural network parameters to generate an initial latent representation that characterizes the initial state of the environment, wherein the initial latent representation is in a latent space;

generating a trajectory of latent representations that comprises a sequence of multiple latent representations that are in the latent space and that each characterize a different state of the environment, wherein the sequence of multiple latent representations start with the initial latent representation that characterizes the initial state of the environment, and wherein generating the trajectory of latent representations comprises, at each subsequent step that is after a beginning step in a plurality of steps:

processing a current latent representation that is generated by using a transition neural network and without processing any observation that characterizes any state of the environment using the policy neural network and in accordance with current values of the plurality of policy neural network parameters to generate an action selection output;

selecting, using the action selection output, an action that would be performed by the agent when the environment is in a current state characterized by the current latent representation; and

processing the current latent representation that is generated by using the transition neural network and the action that is selected using the policy neural network and without processing any observation that characterizes any state of the environment using the transition neural network having a plurality of transition neural network parameters to generate an updated latent representation that is in the latent space and that characterizes a next state that the environment would transition into if the agent performed the selected action when the environment is in the current state characterized by the current latent representation;

for each latent representation in the sequence of multiple latent representations included in the trajectory that are in the latent space:

determining a predicted reward; and

processing the latent representation and without processing any observation that characterizes any state of the environment using a value neural network having a plurality of value neural network parameters and in accordance with current values of the plurality of value neural network parameters to generate a predicted state value that represents a time-adjusted sum of predicted rewards that would be received by the agent by selecting actions using action selection outputs starting from a state characterized by the latent representation;

determining a target state value for each latent representation in the trajectory from the predicted rewards and the predicted state values for the latent representations in the trajectory;

training, based on the predicted state values that have been generated by using the value neural network from the latent representations in the latent space, the policy neural network to update the current values of the plurality of policy neural network parameters, wherein training the policy neural network comprises determining an update to the current values of the plurality of policy neural network parameters that encourages the policy neural network to generate action selection outputs that result in actions being selected that improve the predicted state values; and

training, based on the predicted state values that have been generated by using the value neural network from the latent representations in the latent space, the value neural network to update the current values of the plurality of value neural network parameters, wherein training the value neural network comprises determining an update to the current values of the plurality of value neural network parameters by optimizing a value objective function that measures a difference between the predicted state values and the target state values.

19 . A non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations for training a policy neural network having a plurality of policy neural network parameters used to select actions to be performed by an agent to control the agent in an environment, the operations comprising:

obtaining an initial observation that characterizes an initial state of the environment;

processing at least the initial observation by using a representation neural network having a plurality of representation neural network parameters to generate an initial latent representation that characterizes the initial state of the environment, wherein the initial latent representation is in a latent space;

generating a trajectory of latent representations that comprises a sequence of multiple latent representations that are in the latent space and that each characterize a different state of the environment, wherein the sequence of multiple latent representations start with the initial latent representation that characterizes the initial state of the environment, and wherein generating the trajectory of latent representations comprises, at each subsequent step that is after a beginning step in a plurality of steps:

processing a current latent representation that is generated by using a transition neural network and without processing any observation that characterizes any state of the environment using the policy neural network and in accordance with current values of the plurality of policy neural network parameters to generate an action selection output;

selecting, using the action selection output, an action that would be performed by the agent when the environment is in a current state characterized by the current latent representation; and

processing the current latent representation that is generated by using the transition neural network and the action that is selected using the policy neural network and without processing any observation that characterizes any state of the environment using the transition neural network having a plurality of transition neural network parameters to generate an updated latent representation that is in the latent space and that characterizes a next state that the environment would transition into if the agent performed the selected action when the environment is in the current state characterized by the current latent representation;

for each latent representation in the sequence of multiple latent representations included in the trajectory that are in the latent space:

determining a predicted reward; and

processing the latent representation and without processing any observation that characterizes any state of the environment using a value neural network having a plurality of value neural network parameters and in accordance with current values of the plurality of value neural network parameters to generate a predicted state value that represents a time-adjusted sum of predicted rewards that would be received by the agent by selecting actions using action selection outputs starting from a state characterized by the latent representation;

determining a target state value for each latent representation in the trajectory from the predicted rewards and the predicted state values for the latent representations in the trajectory;

training, based on the predicted state values that have been generated by using the value neural network from the latent representations in the latent space, the policy neural network to update the current values of the plurality of policy neural network parameters, wherein training the policy neural network comprises determining an update to the current values of the plurality of policy neural network parameters that encourages the policy neural network to generate action selection outputs that result in actions being selected that improve the predicted state values; and

training, based on the predicted state values that have been generated by using the value neural network from the latent representations in the latent space, the value neural network to update the current values of the plurality of value neural network parameters, wherein training the value neural network comprises determining an update to the current values of the plurality of value neural network parameters by optimizing a value objective function that measures a difference between the predicted state values and the target state values.

20 . The method of claim 1 , wherein the agent is a mechanical agent and the environment is a real-world environment.

21 . The method of claim 20 , wherein the agent is a robot.

22 . The method of claim 1 , wherein the environment is a real-world environment of a service facility comprising a plurality of items of electronic equipment and the agent is an electronic agent configured to control operation of the service facility.

23 . The method of claim 1 , wherein the environment is a real-world manufacturing environment for manufacturing a product and the agent comprises an electronic agent configured to control a manufacturing unit or a machine that operates to manufacture the product.

24 . The method of claim 1 , wherein:

the current observation comprises high-dimensional sensor data; and

the latent representation is a lower-dimensional representation of the current observation.

25 . The system of claim 18 , wherein:

the current observation comprises high-dimensional sensor data; and

the latent representation is a lower-dimensional representation of the current observation.

Assignments (1)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICATION NUMBER 16/953,977 PREVIOUSLY RECORDED AT REEL: 054631 FRAME: 0307. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Feb 23, 2021
From: HAFNER, DANIJAR; NOROUZI, MOHAMMAD; LILLICRAP, TIMOTHY PAUL
To: GOOGLE LLC
Reel/Frame 055436/0018 →
Continuity (2)
Provisional Application 62941538 · Nov 27, 2019
Related Publication 20210158162A1 · May 27, 2021
References Cited (76)
US 20190244099A1 · Schaul · 2019 [cited by examiner]
US 20190354867A1 · Czarnecki · 2019 [cited by examiner]
US 20190354869A1 · Warde-Farley · 2019 [cited by examiner]
US 20200265305A1 · Budden · 2020 [cited by examiner]
US 20200293883A1 · Budden · 2020 [cited by examiner]
US 20200372366A1 · Badia · 2020 [cited by examiner]
US 20210034970A1 · Soyer · 2021 [cited by examiner]
WO WO2018153807A1 · 2018 [cited by examiner]
Tingwu Wang et al., “Exploring Model-based Planning with Policy Networks,” https://arxiv.org/abs/1906.08649. (Year: 2019). [cited by examiner]
Pengyu Cheng et al, “Straight-Through Estimator as Projected Wasserstein Gradient Flow,” (Year: 2019). [cited by examiner]
Alemi et al., “Deep variational information bottleneck,” arXiv preprint arXiv:1612.00410, Dec. 2016, 19 pages. [cited by applicant]
Banijamali et al., “Robust locally-linear controllable embedding,” International Conference on Artificial Intelligence and Statistics, Mar. 2018, 1751-1759. [cited by applicant]
Barth-Maron et al., “Distributed distributional deterministic policy gradients,” arXiv preprint arXiv:1804.08617, Apr. 2018, 16 pages. [cited by applicant]
Beattie et al., “Deepmind lab,” arXiv preprint arXiv:1612.03801, Dec. 2016, 11 pages. [cited by applicant]
Bellemare et al., “The arcade learning environment: An evaluation platform for general agents,” Journal of Artificial Intelligence Research, Jun. 2013, 47:253-79. [cited by applicant]
Buckman et al., “Sample-efficient reinforcement learning with stochastic ensemble value expansion,” Advances in Neural Information Processing Systems, 2018, 31:8224-34. [cited by applicant]
Buesing et al., “Learning and querying fast generative models for reinforcement learning,” arXiv preprint arXiv:1802.03006, Feb. 2018, 15 pages. [cited by applicant]
Byravan et al., “Imagined value gradients: Model-based policy optimization with tranferable latent dynamics models,” Conference on Robot Learning, May 2020, 566-589. [cited by applicant]
Castro et al., “Dopamine: A research framework for deep reinforcement learning,” arXiv preprint arXiv:1812.06110, Dec. 2018, 22 pages. [cited by applicant]
Chua et al., “Deep reinforcement learning in a handful of trials using probabilistic dynamics models,” Advances in Neural Information Processing Systems, 2018, 4754-4765. [cited by applicant]
Clevert et al., “Fast and accurate deep network learning by exponential linear units,” arXiv preprint arXiv:1511.07289, Nov. 2015, 14 pages. [cited by applicant]
Dillon et al., “Tensorflow distributions,” arXiv preprint arXiv:1711.10604, Nov. 2017, 13 pages. [cited by applicant]
Doerr et al., “Probabilistic recurrent state-space models,” arXiv preprint arXiv:1801.10395, Jan. 2018, 16 pages. [cited by applicant]
Ebert et al., “Self-supervised visual planning with temporal skip connections,” arXiv preprint arXiv:1710.05268, Oct. 2017, 13 pages. [cited by applicant]
Eslami et al., “Neural scene representation and rendering,” Science, Jun. 2018, 360(6394):1204-10. [cited by applicant]
Espeholt et al., “Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures,” arXiv preprint arXiv:1802.01561, Feb. 2018, 22 pages. [cited by applicant]
Feinberg et al., “Model-based value estimation for efficient model-free reinforcement learning,” arXiv preprint arXiv:1803.00101, Feb. 2018, 12 pages. [cited by applicant]
Foerster et al., “Dice: The infinitely differentiable monte-carlo estimator,” arXiv preprint arXiv:1802.05098, Feb. 2018, 10 pages. [cited by applicant]
Gelada et al., “Deepmdp: Learning continuous latent space models for representation learning,” arXiv preprint arXiv:1906.02736, Jun. 2019, 31 pages. [cited by applicant]
Gregor et al., “Shaping belief states with generative environment models for rl,” Advances in Neural Information Processing Systems, 2019, 32:13475-87. [cited by applicant]
Guo et al., “Neural predictive belief representations,” arXiv preprint arXiv:1811.06407, Nov. 2018, 15 pages. [cited by applicant]
Gutmann et al., “Noise-contrastive estimation: A new estimation principle for unnormalized statistical models,” Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Mar. 2010… [cited by applicant]
Ha et al., “World models,” arXiv preprint arXiv:1803.10122, Mar. 2018, 21 pages. [cited by applicant]
Hafner et al., “Dream to control: Learning behaviors by latent imagination,” arXiv preprint arXiv:1912.01603, Dec. 2019, 20 pages. [cited by applicant]
Hafner et al., “Learning latent dynamics for planning from pixels,” International Conference on Machine Learning, May 201, 2555-2565. [cited by applicant]
Heess et al., “Learning continuous control policies by stochastic value gradients,” Advances in Neural Information Processing Systems, 2015, 2944-2952. [cited by applicant]
Henaff et al., “Model-based planning with discrete and continuous actions,” arXiv preprint arXiv:1705.07177, May 2017, 11 pages. [cited by applicant]
Henaff et al., “Model-predictive policy learning with uncertainty regularization for driving in dense traffic.” arXiv preprint arXiv:1901.02705, Jan. 2019, 19 pages. [cited by applicant]
Hessel et al., “Rainbow: Combining improvements in deep reinforcement learning,” Thirty-Second AAAI Conference on Artificial Intelligence, Apr. 2018, 8 pages. [cited by applicant]
Jaderberg et al., “Reinforcement learning with unsupervised auxiliary tasks,” arXiv preprint arXiv:1611.05397, Nov. 2016, 14 pages. [cited by applicant]
Jordan et al., “An introduction to variational methods for graphical models,” Learning in graphical models, 1998, 105-161. [cited by applicant]
Kaiser et al., “Model-based reinforcement learning for atari,” arXiv preprint arXiv:1903.00374, Mar. 2019, 28 pages. [cited by applicant]
Kalman et al., “A new approach to linear filtering and prediction problems,” Journal of Fluids Engineering, Mar. 1960, 11 pages. [cited by applicant]
Karl et al., “Deep variational bayes filters: Unsupervised learning of state space models from raw data,” arXiv preprint arXiv:1605.06432, May 2016, 13 pages. [cited by applicant]
Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, Dec. 2014, 15 pages. [cited by applicant]
Kingma et al., “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, Dec. 2013, 14 pages. [cited by applicant]
Krishnan et al., “Deep kalman filters,” arXiv preprint arXiv:1511.05121, Nov. 2015, 17 pages. [cited by applicant]
Kurutach et al., “Model-ensemble trust-region policy optimization,” arXiv preprint arXiv:1802.10592, Feb. 2018, 15 pages. [cited by applicant]
LeCun et al., “Backpropagation applied to handwritten zip code recognition,” Neural computation, Dec. 1989, 1(4):541-51. [cited by applicant]
Lee et al., “Stochastic latent actor-critic: Deep reinforcement learning with a latent variable model,” Advances in Neural Information Processing Systems, 2020, 12 pages. [cited by applicant]
Lillicrap et al., “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, Sep. 2015, 14 pages. [cited by applicant]
Lowrey et al., “Plan online, learn offline: Efficient learning and exploration via model-based control,” arXiv preprint arXiv:1811.01848, Nov. 2018, 15 pages. [cited by applicant]
Machado et al., “Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents,” Journal of Artificial Intelligence Research, Mar. 2018, 61:523-62. [cited by applicant]
McAllester et al., “International Conference on Artificial Intelligence and Statistics,” PMLR, Jun. 2020, 875-884. [cited by applicant]
Mnih et al., “Asynchronous methods for deep reinforcement learning,” International Conference on Machine Learning, Jun. 2016, 1928-1937. [cited by applicant]
Mnih et al., “Human-level control through deep reinforcement learning,” Nature, Feb. 2015, 518(7540):529-33. [cited by applicant]
Oh et al., “Value prediction network,” Advances in Neural Information Processing Systems, 2017, 30:6118-28. [cited by applicant]
Oord et al., “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748, Jul. 2018, 13 pages. [cited by applicant]
Parmas et al., “PIPPS: Flexible model-based policy search robust to the curse of chaos,” International Conference on Machine Learning, Jul. 2018, 4065-4074. [cited by applicant]
Poole et al., “On variational bounds of mutual information,” arXiv preprint arXiv:1905.06922, May 2019, 14 pages. [cited by applicant]
Rezende et al., “Stochastic backpropagation and approximate inference in deep generative models,” arXiv preprint arXiv:1401.4082, Jan. 2014, 14 pages. [cited by applicant]
Schrittwieser et al., “Mastering atari, go, chess and shogi by planning with a learned model,” arXiv preprint arXiv:1911.08265, Nov. 2019, 21 pages. [cited by applicant]
Schulman et al., “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, Jul. 2017, 12 pages. [cited by applicant]
Silver et al., “Deterministic policy gradient algorithms,” Proceedings of the 31st International Conference on Machine Learning, 2014, 9 pages. [cited by applicant]
Silver et al., “Mastering the game of go without human knowledge,” nature, Oct. 2017, 550(7676):354-9. [cited by applicant]
Srinivas et al., “Universal planning networks,” arXiv preprint arXiv:1804.00645, Apr. 2018, 21 pages. [cited by applicant]
Sutton et al., “Dyna, an integrated architecture for learning, planning, and reacting,” ACM Sigart Bulletin, Jul. 1991, 2(4):160-3. [cited by applicant]
Sutton et al., “Reinforcement learning: An introduction,” MIT press, Oct. 2018, 10 pages. [cited by applicant]
Tassa et al., “Deepmind control suite,” arXiv preprint arXiv:1801.00690, Jan. 2018, 24 pages. [cited by applicant]
Tishby et al., “The information bottleneck method,” arXiv preprint physics0004057, Apr. 2000, 16 pages. [cited by applicant]
Wang et al., “Benchmarking model-based reinforcement learning,” CoRR abs1907.02057, 2019, 25 pages. [cited by applicant]
Wang et al., “Exploring model-based planning with policy networks,” arXiv preprint arXiv:1906.08649, Jun. 2019, 20 pages. [cited by applicant]
Watter et al., “Embed to control: A locally linear latent dynamics model for control from raw images,” Advances in neural information processing systems, 2015, 2746-2754. [cited by applicant]
Weber et al., “Imagination-augmented agents for deep reinforcement learning,” arXiv preprint arXiv:1707.06203, Jul. 2017. [cited by applicant]
Williams et al., “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Machine learning, May 1992, 8(3-4):229-56. [cited by applicant]
Zhang et al., “Solar: Deep structured representations for model-based reinforcement learning,” International Conference on Machine Learning, May 2019, 7444-7453. [cited by applicant]
Cited By (1)
US 12,676,143