Reinforcement learning based ground vehicle control techniques

1. A method comprising:

training a reinforcement learning controller based on alternating behavior training data and target training data for a simulated ground vehicle environment, during a simulation mode;

further training the simulation trained reinforcement learning controller based on a ground vehicle environment, during an operating mode of a ground vehicle; and

outputting an action based on a current state of the ground vehicle environment utilizing the simulation and environment trained reinforcement learning controller, during the operating mode of the ground vehicle.

2. The method according to claim 1 , wherein training the reinforcement learning controller includes:

optimizing the reinforcement learning controller for composite energy efficiency and drivability costs.

3. The method according to claim 1 , wherein training the reinforcement learning controller, during the simulation mode, includes:

alternatingly generating the behavioral training data by both a guide controller and the reinforcement learning controller, wherein the guide controller generates the training data to be used for off-policy training of the reinforcement learning controller under a model-free reinforcement learning policy or an actor-critic model free reinforcement learning setting;

collecting the training data in a buffer; and

training the reinforcement learning controller on the training data collected in the buffer.

4. The method according to claim 3 , wherein training the reinforcement learning controller, during the simulation mode, includes:

running simulation using the guide controller to generate initial training data to be used for off-policy training of the reinforcement learning controller under a model-free reinforcement learning policy.

5. The method according to claim 3 , wherein training the model-free reinforcement learning policy, during the simulation mode, includes:

intermittently running simulations utilizing a behavioral model-free reinforcement learning policy to generate off-policy training data.

6. The method according to claim 3 , wherein training the model-free reinforcement learning policy, during the simulation mode, includes:

running simulation utilizing a guiding stable policy to generate training episode data, wherein the model-free reinforcement learning policy is trained in a stable environment region.

7. The method according to claim 1 , wherein further training the simulation trained reinforcement learning controller, during the operating mode, includes:

training the reinforcement learning controller including modifying one or more parameters of a controller of the ground vehicle environment, wherein the one or more modified parameters are utilized by the controller of the ground vehicle environment and the reinforcement learning controller.

8. The method according to claim 1 , wherein training the reinforcement learning controller comprises iteratively:

receiving a current state;

determining an action-value function associating a return to a given state and action pairing based on the current state received in response to a previous action; and

updating a policy with an action value function providing a best return for a given state and action pairing.

9. The method according to claim 8 , wherein:

states include one or more of a group consisting of a plurality of powertrain operating parameter states, and a plurality of driving environment input states;

actions include one or more of a group consisting of a plurality of powertrain operating parameter actions, and a plurality of driver control input actions.

10. The method according to claim 1 , further comprising decoupling the reinforcement learning controller from data sources.

11. The method according to claim 10 , decoupling the reinforcement learning controller from data sources includes:

grouping similar data sources into a plurality of groupings, each grouping capturing a different state symantec.

12. A ground vehicle control system comprising:

a plurality of sensors configured to detect a plurality of ground vehicle operating parameters, driver control inputs, and one or more driving environment parameters, wherein the one or more driving environment parameters includes at least one predicted driving environment parameter; and

a predictive enhanced cruise controller including a reinforcement learning controller configured to;

train the reinforcement learning controller based on alternating behavior training data and target training data for a simulated ground vehicle environment, during a simulation mode;

further train the simulation trained reinforcement learning controller based on a ground vehicle environment, during an operating mode of a ground vehicle; and

output an action based on a current state of the ground vehicle environment utilizing the simulation and environment trained reinforcement learning controller, during the operating mode of the ground vehicle.

13. The ground vehicle control system of claim 12 , wherein the reinforcement learning controller is further configured to;

optimize the reinforcement learning controller for composite energy efficiency and drivability costs.

14. The ground vehicle control system of claim 12 , wherein the reinforcement learning controller is further configured to;

alternatingly generate the behavioral training data by both a guide controller and the reinforcement learning controller;

collect the training data in a buffer; and

train the reinforcement learning controller on the training data collected in the buffer.

15. The ground vehicle control system of claim 12 , wherein the reinforcement learning controller is further configured to;

generate initial training data by the guide controller; and

train by the reinforcement controller on the initial training data.

16. The ground vehicle control system of claim 12 , wherein the reinforcement learning controller is further configured to;

train the reinforcement learning controller, wherein an output of the reinforcement learning controller modifies one or more outputs of a controller of the ground vehicle environment.

17. The ground vehicle control system of claim 12 , wherein the reinforcement learning controller includes:

an action-value (Q) function engine configured to receive a current state of the ground vehicle operating environment and determine actions and expected returns for the actions;

a policy (π) engine configure to determine a current action for the current state that provides a maximum expected return and output the determined current action;

an optimizer engine configured to optimize the functions of the actions and expected returns of the action-value (Q) function engine and the current action for the current state that provide the best return of the policy (π) engine, wherein the best return is a composite of energy efficiency, powertrain control and drivability functions.

18. The ground vehicle control system of claim 12 , wherein:

states include one or more of a group consisting of a plurality of powertrain operating parameter states, and a plurality of driving environment input states;

actions include one or more of a group consisting of a plurality of powertrain operating parameter actions, and a plurality of driver control input actions.

19. The ground vehicle control system of claim 12 , wherein the reinforcement learning controller is further configured to;

decouple the reinforcement learning controller from data sources.

20. The ground vehicle control system of claim 19 , wherein the reinforcement learning controller groups similar data sources into a plurality of groupings, each grouping capturing a different state symantec.