Reinforcement and model learning for vehicle operation

1. A method for use in traversing a vehicle transportation network, the method comprising:

traversing, by an autonomous vehicle, a vehicle transportation network, wherein traversing the vehicle transportation network includes:

determining a route of the autonomous vehicle within the vehicle transportation network;

executing the route of the autonomous vehicle;

detecting an operational scenario based on the route of the autonomous vehicle and a location of the autonomous vehicle;

determining a particular scenario-specific operation control evaluation module based on the operational scenario, wherein the scenario-specific operation control evaluation module includes models that determine a candidate vehicle control action based on an operational environment of the autonomous vehicle, wherein the models include an exploration model based on selecting a low probability action in a semi-random manner and an exploitation model based on selecting a high probability action;

instantiating a scenario-specific operational control evaluation module instance based on the particular scenario-specific operation control evaluation module;

traversing a portion of the vehicle transportation network by executing the candidate vehicle control action using the exploration model or the exploitation model;

observing a state resulting from the execution of the candidate vehicle control action;

updating the scenario-specific operational control evaluation module instance based on the state;

generating a state-action history entry based on the candidate vehicle control action and the state; and

storing the state-action history entry in a scenario-specific operation control database.

2. The method of claim 1 , wherein the exploration model and the exploitation model apply a stored state-action history.

3. The method of claim 1 , wherein the exploitation model is based on selecting the candidate vehicle control action using the particular scenario-specific operation control evaluation module.

4. The method of claim 1 , further comprising:

determining that the operational scenario is not complete; and

determining a second particular scenario-specific operation control evaluation module based on the operational scenario.

5. The method of claim 1 , further comprising:

determining that the operational scenario is complete;

determining driver-initiated vehicle control actions, wherein a number of driver-initiated vehicle control actions is greater than a threshold; and

creating a customized operational scenario based on the observed state and the driver-initiated vehicle control actions.

6. The method of claim 1 , wherein the state is associated with an operational environment.

7. The method of claim 1 , wherein the state is associated with a vehicle state of the autonomous vehicle.

8. The method of claim 1 , wherein the state is associated with a vehicle state of another vehicle in the vehicle transportation network.

9. The method of claim 1 further comprising:

solving a second operational scenario based on the stored state-action history.

10. An autonomous vehicle comprising:

a processor configured to execute instructions stored on a non-transitory computer readable medium to:

determine a route of the autonomous vehicle within a vehicle transportation network;

execute the route of the autonomous vehicle;

detect an operational scenario based on the route of the autonomous vehicle and a location of the autonomous vehicle;

determine a particular scenario-specific operation control evaluation module based on the operational scenario, wherein the scenario-specific operation control evaluation module includes a model that determines a candidate vehicle control action based on an operational environment of the autonomous vehicle, wherein the model is based on a low probability random selection of the candidate vehicle control action;

instantiate a scenario-specific operational control evaluation module instance based on the particular scenario-specific operation control evaluation module;

traverse a portion of the vehicle transportation network based on an execution of the candidate vehicle control action using the model;

observe a state resulting from the execution of the candidate vehicle control action; and

generate a state-action history entry based on the candidate vehicle control action and the state; and

a memory configured to store the state-action history entry in a scenario-specific operation control database.

11. The autonomous vehicle of claim 10 , wherein the model applies the stored state-action history such that a probability of the random selection of the candidate vehicle control action diminishes over time proportional to a volume of the stored state-action history.

12. The autonomous vehicle of claim 10 , wherein the model is based on selecting the candidate vehicle control action using the particular scenario-specific operation control evaluation module.

13. The autonomous vehicle of claim 10 , wherein the processor is further configured to execute instructions stored on the non-transitory computer readable medium to:

determine that the operational scenario is not complete; and

determine a second particular scenario-specific operation control evaluation module based on the operational scenario.

14. The autonomous vehicle of claim 10 , wherein the processor is further configured to execute instructions stored on the non-transitory computer readable medium to:

determine that the operational scenario is complete;

determine driver-initiated vehicle control actions, wherein a number of driver-initiated vehicle control actions is greater than a threshold; and

create a customized operational scenario based on the observed state and the driver-initiated vehicle control actions.

15. The autonomous vehicle of claim 10 , wherein the processor is further configured to execute instructions stored on the non-transitory computer readable medium to:

solve a second operational scenario based on the stored state-action history.