Belief state determination for real-time decision-making

1. An apparatus for real-time decision-making for a vehicle using belief state determination, the apparatus comprising:

a processor configured to:

receive, while the vehicle is traversing a vehicle transportation network, operational environment data, wherein the operational environment data includes data associated with an object external to the vehicle;

establish, using an operational environment monitor, an observation that relates the object to a distinct vehicle operation scenario;

compute, using a belief state model of the operational environment monitor, a belief state for the observation directly from the operational environment data;

provide, from the operational environment monitor, the computed belief state to a decision component implementing a policy that maps a respective belief state for the object within the distinct vehicle operation scenario to a respective candidate vehicle control action;

receive, from the policy of the decision component, a candidate vehicle control action; and

select, for traversing the vehicle transportation network, a vehicle control action from any available candidate vehicle control actions.

2. The apparatus of claim 1 , wherein the belief state model of the operational environment monitor excludes a belief state update equation.

3. The apparatus of claim 1 , wherein:

the distinct vehicle operation scenario comprises an intersection of the vehicle transportation network;

the observation comprises a set of observations including an observation that the object is at the intersection, an observation that the object is inside the intersection, an observation that the object is approaching the intersection, and an observation that the object is past the intersection; and

to compute the belief state for the observation comprises to compute a set of belief states, each corresponding to a respective observation of the set of observations.

4. The apparatus of claim 1 , wherein:

the decision component is a scenario-specific operational control evaluation model of the distinct vehicle operation scenario that, when solved, results in the policy; and

the processor is configured to:

provide, from the operational environment monitor, the observation to the scenario-specific operational control evaluation model; and

solve the scenario-specific operational control evaluation model while the vehicle is traversing the vehicle transportation network, wherein to solve the scenario-specific operational control evaluation model comprises to:

update a previous belief state maintained by the scenario-specific operational control evaluation model to an updated belief state using a belief state update equation and the observation; and

if one of the observation is invalid or a difference between the updated belief state and the computed belief state is greater than a threshold difference, use the computed belief state in place of the updated belief state.

5. The apparatus of claim 1 , wherein:

the decision component is a scenario-specific operational control evaluation model of the distinct vehicle operation scenario that, when solved, results in the policy; and

the processor is configured to:

provide, from the operational environment monitor, the observation to the scenario-specific operational control evaluation model;

select a proper subset of belief states from a set of available belief states for the distinct vehicle operation scenario; and

solve the scenario-specific operational control evaluation model to determine the policy while the vehicle is traversing the vehicle transportation network, responsive to the observation, using only the proper subset of belief states.

6. The apparatus of claim 1 , wherein the policy is pre-defined by:

selecting a proper subset of belief states from a set of available belief states for the distinct vehicle operation scenario; and

solving a scenario-specific operational control evaluation model of the distinct vehicle operation scenario using the proper subset of belief states to define the policy.

7. The apparatus of claim 1 , wherein the policy is represented by a finite state controller, and the computed belief state maps to a node of the finite state controller.

8. The apparatus of claim 1 , wherein:

the decision component comprises a Partially Observable Markov Decision Process model.

9. The apparatus of claim 1 , wherein the operational environment monitor comprises a blocking monitor configured to compute the belief state as a probability that the object will block a predicted path of the vehicle.

10. A method for real-time decision-making for a vehicle using belief state determination, the method comprising:

receiving, while the vehicle is traversing a vehicle transportation network, operational environment data, wherein the operational environment data includes data associated with an object external to the vehicle;

establishing, using an operational environment monitor, an observation that relates the object to a distinct vehicle operation scenario;

computing, using a belief state model of the operational environment monitor, a belief state for the observation directly from the operational environment data;

providing, from the operational environment monitor, the computed belief state to a decision component implementing a policy that maps a respective belief state for the object within the distinct vehicle operation scenario to a respective candidate vehicle control action;

receiving, from the policy of the decision component, a candidate vehicle control action; and

selecting, for traversing the vehicle transportation network, a vehicle control action from any available candidate vehicle control actions.

11. The method of claim 10 , wherein the belief state model of the operational environment monitor excludes a belief state update equation.

12. The method of claim 10 , wherein:

the operational environment monitor comprises a blocking monitor; and

computing the belief state comprises computing the belief state as a probability that the object will block a predicted path of the vehicle.

13. The method of claim 10 , wherein:

the decision component is a scenario-specific operational control evaluation model of the distinct vehicle operation scenario that, when solved, results in the policy; and the method comprises:

providing, from the operational environment monitor, the observation to the scenario-specific operational control evaluation model; and

solving the scenario-specific operational control evaluation model while the vehicle is traversing the vehicle transportation network, wherein solving the scenario-specific operational control evaluation model comprises:

updating a previous belief state maintained by the scenario-specific operational control evaluation model to an updated belief state using a belief state update equation; and

if one of the observation is invalid or a difference between the updated belief state and the computed belief state is greater than a threshold difference, using the computed belief state in place of the updated belief state, and otherwise using the updated belief state.

14. The method of claim 10 , wherein:

the decision component is a scenario-specific operational control evaluation model of the distinct vehicle operation scenario that, when solved, results in the policy; and the method comprises:

providing, from the operational environment monitor, the observation to the scenario-specific operational control evaluation model;

selecting a proper subset of belief states from a set of available belief states for the distinct vehicle operation scenario; and

solving the scenario-specific operational control evaluation model to determine the policy while the vehicle is traversing the vehicle transportation network, responsive to the observation, using only the proper subset of belief states.

15. The method of claim 10 , further comprising:

pre-defining the policy by:

selecting a proper subset of belief states from a set of available belief states for the distinct vehicle operation scenario; and

solving a scenario-specific operational control evaluation model of the distinct vehicle operation scenario using the proper subset of belief states to define the policy.

16. The method of claim 10 , wherein the policy is represented by a finite state controller, and the computed state belief maps to a node of the finite state controller.

17. A vehicle implementing real-time decision-making using belief state determination, the vehicle comprising:

a processor configured to:

receive, while the vehicle is traversing a vehicle transportation network, operational environment data, wherein the operational environment data includes data associated with an object external to the vehicle;

establish, using an operational environment monitor, an observation that relates the object to a distinct vehicle operation scenario;

compute, using a belief state model of the operational environment monitor, a belief state for the observation directly from the operational environment data;

provide, from the operational environment monitor, the computed belief state to a decision component implementing a policy that maps a respective belief state for the object within the distinct vehicle operation scenario to a respective candidate vehicle control action;

receive, from the policy of the decision component, a candidate vehicle control action; and

select, for traversing the vehicle transportation network, a vehicle control action from any available candidate vehicle control actions.

18. The vehicle of claim 17 , wherein the belief state model of the operational environment monitor excludes a belief state update equation.

19. The vehicle of claim 17 , wherein:

the decision component is a scenario-specific operational control evaluation model of the distinct vehicle operation scenario that, when solved, results in the policy; and

the processor is configured to:

provide, from the operational environment monitor, the observation to the scenario-specific operational control evaluation model; and

solve the scenario-specific operational control evaluation model while the vehicle is traversing the vehicle transportation network, wherein to solve the scenario-specific operational control evaluation model comprises to:

update a previous belief state maintained by the scenario-specific operational control evaluation model to an updated belief state using a belief state update equation; and

if one of the observation is invalid or a difference between the updated belief state and the computed belief state is greater than a threshold difference, use the computed belief state in place of the updated belief state.

20. The vehicle of claim 17 , further comprising:

a trajectory controller configured to execute the vehicle control action.