Granted Patent B2

US 11,565,717 · App. 17/826,655 · Granted Jan 31, 2023

Inventors: Melinda Kothbauer (Ann Arbor, MI); Jeff Sterniak (Ann Arbor, MI); Edwin B. Olson (Ann Arbor, MI)

Assignee: May Mobility, Inc.

B60W60/0011B60W40/02G05D1/0011B60W2554/20B60W2554/4041B60W2556/45

Abstract

Claims (54)

1. A method comprising:

autonomously controlling a vehicle with an onboard computing system of the vehicle, comprising, at each election step of an election cycle:

determining a vehicle state estimate based on a set of sensor inputs;

determining a set of electable policies based on the vehicle state estimate;

selecting a policy from the set of electable policies;

determining vehicle instructions based on the policy; and

controlling the vehicle according to the vehicle instructions; and

while autonomously controlling the vehicle:

based on the vehicle state estimate, determining satisfaction of an event trigger, wherein the event trigger comprises a probability threshold of an impasse occurrence along a target vehicle path;

based on the event trigger satisfaction, providing a set of outputs to a tele-assist platform; and

updating the set of electable policies to include a tele-operator-approved policy determined with the tele-assist platform.

2. The method of claim 1 , further comprising performing a set of multiple simulations, wherein an individual policy is autonomously selected from the set of electable policies based on a set of outcomes resulting from the set of multiple simulations.

3. The method of claim 2 , further comprising estimating a set of state estimates for a set of dynamic objects in an environment of the vehicle based on the set of sensor inputs, wherein each of the set of multiple simulations is performed based on the set of state estimates and the vehicle state estimate.

4. The method of claim 1 , wherein the vehicle state estimate comprises a vehicle position estimate, and wherein the method further comprises determining a scenario, wherein the set of electable policies are determined by:

determining a set of policies from a predetermined set based on the vehicle position estimate; and

refining the set of policies based on the scenario.

5. The method of claim 1 , further comprising: determining an autonomously unelectable policy with the onboard computing system, wherein the set of outputs provided to the tele-assist platform comprises the autonomously unelectable policy.

6. The method of claim 5 , further comprising: after providing the set of outputs to the tele-assist platform, receiving tele-operator approval of the autonomously unelectable policy from the tele-assist platform and updating the set of electable policies in response.

7. The method of claim 1 , wherein the policy is selected based on a target vehicle path, wherein the method further comprises updating the target vehicle path based on a tele-operator-approved path received from the tele-assist platform.

8. The method of claim 7 , wherein the tele-operator-approved path comprises a set of waypoints which circumvent an obstacle.

9. The method of claim 8 , wherein the set of outputs provided to the tele-assist platform comprises a set of estimated object parameters associated with the obstacle.

10. The method of claim 8 , wherein after updating the target vehicle path based on a tele-operator-approved path, autonomously controlling the vehicle comprises: selecting an updated policy from the set of electable policies, wherein the updated policy deviates from the tele-operator-approved path.

11. The method of claim 10 , wherein the updated policy is associated with a return to the target vehicle path.

12. The method of claim 1 , wherein the event trigger further corresponds to a time threshold.

13. The method of claim 1 , wherein the tele-operator approved policy is received via a wireless connection between the onboard computing system and the tele-assist platform with a latency greater than 50 milliseconds.

14. A method for tele-operation of an autonomous agent, comprising:

at the autonomous agent, determining a first set of inputs;

based on the first set of inputs, determining satisfaction of an event trigger associated with an impasse along a target path;

based on the event trigger satisfaction, providing a set of outputs to a tele-assist platform, the set of outputs comprising a proposed policy which deviates from the target path;

receiving a second set of inputs from the tele-assist platform, the second set of inputs comprising a tele-operator approval of the proposed policy;

based on the first and second sets of inputs, determining a set of electable policies comprising the proposed policy based on the first and second sets of inputs;

at the autonomous agent, selecting a policy from the set of electable policies based on the target path; and

autonomously controlling the vehicle based on the policy.

15. The method of claim 14 , further comprising: prior to selecting the policy, updating the target path based on the second set of inputs.

16. The method of claim 15 , wherein the second set of inputs comprise a set of waypoints.

17. The method of claim 14 , wherein the selected policy is different from the proposed policy.

18. The method of claim 15 , wherein the first set of inputs comprises a position estimate and a scenario, wherein the set of electable policies are determined by:

determining a set of policies from a labeled map based on the position estimate;

expanding the set of policies based on the tele-operator approval; and

refining the set of policies to yield the set of electable policies based on the scenario.

19. The method of claim 14 , wherein the second set of inputs comprises a binary decision by a tele-operator.

20. The method of claim 14 , wherein the policy is selected using a multi-policy decision module of the autonomous agent which defines a tele-operator policy decision node and a set of autonomous policy decision nodes downstream of the tele-operator decision node.

21. A method comprising:

autonomously controlling a vehicle with an onboard computing system of the vehicle, comprising, at each election step of an election cycle:

determining a vehicle state estimate based on a set of sensor inputs;

determining a set of electable policies based on the vehicle state estimate;

selecting a policy from the set of electable policies;

determining vehicle instructions based on the policy; and

controlling the vehicle according to the vehicle instructions; and

while autonomously controlling the vehicle:

based on the vehicle state estimate, determining satisfaction of an event trigger;

based on the event trigger satisfaction, providing a set of outputs to a tele-assist platform; and

updating the set of electable policies to include a tele-operator-approved policy determined with the tele-assist platform,

wherein the tele-operator-approved policy is received via a wireless connection between the onboard computing system and the tele-assist platform with a latency greater than 50 milliseconds.

Assignments (2)

Continuity (2)

Cited By (9)

US 12,240,494 US 12,296,849 US 12,319,313 US 12,371,067 US 12,441,364 US 12,559,132 US 12,559,143 US 12,568,191 US 12,594,961