IP Library Granted Patent US 11,016,485
Granted Patent B2
US 11,016,485 · App. 16/367,758 · Granted May 25, 2021

Teleoperation for exception handling

Inventors: Liam Pedersen (San Francisco, CA); Ali Mortazavi (Walnut Creek, CA); Stefan Witwicki (San Carlos, CA); Christopher Ostafew (Mountain View, CA)
Assignees: Nissan North America, Inc.; Renault S.A.S.
G05D1/0061G05D1/0088G05D1/0214G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,016,485
App. No.
16/367,758
Granted
May 25, 2021
Kind
B2
Abstract

Exception handing, such as of obstruction situations, by an autonomous vehicle (AV) is disclosed. A method includes identifying an exception situation; identifying a risk associated with autonomously resolving the exception situation; and in response to the risk exceeding a risk threshold, initiating a request for assistance from a tele-operator, and halting for the tele-operator to respond to the request; and receiving a response from the tele-operator.

Claims (86)

1. A method of exception handling by an autonomous vehicle (AV), the method comprising:

identifying an exception situation in response to detecting a stopped vehicle ahead of the AV;

identifying a risk associated with autonomously resolving the exception situation,

wherein the risk is identified as a weighting of a situation likelihood and a consequence of autonomously resolving the exception situation,

wherein the situation likelihood relates to whether the stopped vehicle is permanently or temporarily stopped and the situation likelihood is determined based on at least two of:

whether hazard lights of the stopped vehicle are flashing,

how long the stopped vehicle has been stopped,

whether the stopped vehicle is at an intersection,

whether pylons are present, or

whether the stopped vehicle was classified as a moving vehicle at a previous time step; and

wherein the consequence relates to a cost of overtaking the stopped vehicle and is determined based on whether the AV has a clear view ahead and sufficient lateral space to overtake the stopped vehicle; and

in response to the risk exceeding a risk threshold:

initiating a request for assistance from a tele-operator; and

halting for the tele-operator to respond to the request; and

receiving a response from the tele-operator.

2. The method of claim 1 , wherein identifying the exception situation comprises:

identifying an obstruction of a first trajectory of the AV;

determining a second trajectory that avoids the obstruction; and

determining that the second trajectory violates a driving norm.

3. The method of claim 2 , further comprising:

in response to the risk not exceeding the risk threshold, operating the AV according to the second trajectory.

4. The method of claim 3 , further comprising:

in response to the received response being a proceed response, operating the AV according to the second trajectory.

5. The method of claim 2 ,

wherein the received response includes a third trajectory, further comprising:

autonomously operating the AV according to the third trajectory.

6. The method of claim 1 , wherein the received response from the tele-operator is selected from a set comprising a first response to wait, a second response to proceed, and a third response that includes a trajectory.

7. The method of claim 1 , further comprising:

identifying another exception situation;

initiating another request for assistance from the tele-operator; and

revoking the another request to the tele-operator.

8. The method of claim 7 , wherein revoking the another request to the tele-operator comprises:

in response to determining that the another exception situation is cleared before receiving a response from the tele-operator to the another request, revoking the another request to the tele-operator.

9. A method of exception handling by an autonomous vehicle (AV), the method comprising:

identifying a lane-obstruction situation that is a stopped vehicle;

identifying a first trajectory for avoiding the lane-obstruction situation; and

in response to identifying, based on the first trajectory, that the lane-obstruction situation is an exception situation:

determining a risk associated with the first trajectory,

wherein the risk is identified as a weighting of a situation likelihood and a consequence of autonomously resolving the exception situation, and

wherein the situation likelihood relates to whether the stopped vehicle is permanently or temporarily stopped and the risk is determined based on at least two of:

whether the stopped vehicle is yielding or is permanently stopped;

whether sensors of the AV can detect oncoming other world objects;

whether a door of the stopped vehicle is open; or

whether, to navigate around the stopped vehicle, the AV would have to travel on an opposite-direction lane; and

in response to determining that the risk exceeds a risk threshold:

initiating a request to a tele-operator, wherein the request includes the first trajectory; and

waiting for a response from the tele-operator.

10. The method of claim 9 , further comprising:

in response to determining that the risk associated with the first trajectory does not exceed the risk threshold:

autonomously operating the AV according to the first trajectory.

11. The method of claim 9 , further comprising:

in response to receiving a proceed response from the tele-operator, operating the AV according to the first trajectory.

12. The method of claim 9 , further comprising:

receiving a second trajectory from the tele-operator; and

operating the AV according to the second trajectory.

13. An apparatus, comprising:

a memory, wherein instructions for exception handling by an autonomous vehicle (AV) are stored in the memory; and

a processor, the processor configured to execute the instructions to:

identify an exception situation;

identify a risk associated with autonomously resolving the exception situation,

wherein the risk is identified as a weighting of a situation likelihood and a consequence of autonomously resolving the exception situation,

wherein the situation likelihood is a measure of understanding, using sensor data from sensors of the AV, of the exception situation and relates to whether an identified person is a policeman or a traffic director, and

wherein the risk weighs the situation likelihood that the identified person is the policeman or the traffic director and the consequence of autonomously resolving the exception situation by ignoring instructions of the policeman or the traffic director; and

in response to the risk exceeding a risk threshold:

initiate a request to a tele-operator;

wait for the tele-operator to respond to the request; and

receive a response from the tele-operator.

14. The apparatus of claim 13 , wherein to identify the exception situation comprises:

identify an obstruction of a first traj ectory of the AV;

determine a second trajectory that avoids the obstruction; and

determine that the second trajectory violates a driving norm.

15. The apparatus of claim 14 , wherein the instructions further comprise instructions to:

in response to the risk not exceeding the risk threshold, autonomously operate the AV according to the second trajectory.

16. The apparatus of claim 15 , wherein the instructions further comprise instructions to:

in response to the received response being a proceed response, autonomously operate the AV according to the second trajectory.

17. The apparatus of claim 14 ,

wherein the received response includes a third trajectory, and

the instructions further comprise instructions to:

autonomously operate the AV according to the third trajectory.

18. The apparatus of claim 13 , wherein the received response from the tele-operator is selected from a set comprising a first response to wait, a second response to proceed, and a third response that includes a trajectory.

19. The apparatus of claim 13 , wherein the instructions further comprise instructions to:

identify another exception situation;

initiate another request for assistance from the tele-operator in response to the another exception situation; and

revoke the another request to the tele-operator.

20. The apparatus of claim 19 , wherein to revoke the another request to the tele-operator comprises:

in response to determining that the another exception situation is cleared before receiving a response to the another request, revoke the another request to the tele-operator.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2021
From: NISSAN NORTH AMERICA, INC.
To: NISSAN MOTOR CO., LTD.
Reel/Frame 057337/0758 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2019
From: NISSAN NORTH AMERICA, INC.
To: NISSAN NORTH AMERICA, INC.; RENAULT S.A.S.
Reel/Frame 050105/0943 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2019
From: PEDERSEN, LIAM; MORTAZAVI, ALI; WITWICKI, STEFAN; OSTAFEW, CHRISTOPHER
To: NISSAN NORTH AMERICA, INC.
Reel/Frame 049060/0913 →
Continuity (1)
Related Publication 20200310417A1 · Oct 1, 2020
Cited By (6)
US 12,358,530 US 12,371,069 US 12,459,535 US 12,482,353 US 12,509,061 US 12,556,660