IP Library Granted Patent US 11,307,597
Granted Patent B2
US 11,307,597 · App. 16/890,506 · Granted Apr 19, 2022

Tele-operation of autonomous cars to negotiate problem situations

Inventors: Liam Pedersen (San Francisco, CA); Maarten Sierhuis (San Francisco, CA); Hans Utz (Campbell, CA); Mauro Della Penna (San Francisco, CA); Terrence Fong (Moffett Field, CA); Mark Allan (Campbell, CA); Maria Bualat (San Jose, CA); Eric Schafer (Kentfield, CA)
Assignees: Nissan North America, Inc.; United States of America as Represented by the Administrator of NASA
G05D1/0291G05D1/0212G05D1/0276G05D1/0287G05D1/0297G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,307,597
App. No.
16/890,506
Granted
Apr 19, 2022
Kind
B2
Abstract

Methods and systems for providing remote support and negotiating problem situations of autonomous operation of vehicles based on signal states and vehicle information are described. A system comprises a memory and a processor configured to execute instructions stored in the memory to: assign vehicles to support queues based on state data, generate a map display including locations of the vehicles, and generate a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data.

Claims (49)

1. A system, comprising:

a memory; and

a processor configured to execute instructions stored in the memory to:

assign vehicles to support queues based on state data,

generate a map display including locations of the vehicles, and

generate a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data.

2. The system of claim 1 , wherein the processor is further configured to execute instructions stored in the memory to:

receive a request from a vehicle for remote support; and

transmit instruction data to the vehicle to provide the remote support.

3. The system of claim 1 , wherein the processor is further configured to execute instructions stored in the memory to:

responsive to autonomous operation of a vehicle operating outside of defined parameter values, transmit instruction data to the vehicle to modify the autonomous operation of the vehicle.

4. The system of claim 1 , wherein the support queues comprise first level control stations, each first level control station receiving the state data for different vehicles.

5. The system of claim 4 , wherein the vehicles are assigned to the first level control stations using a second level control station.

6. The system of claim 1 , wherein the state data includes any of kinetic state data and location data for the vehicles.

7. The system of claim 1 , wherein the processor is further configured to execute instructions stored in the memory to:

responsive to an input signal to a first state indicator representing state data of a vehicle, generate a remote support interface display including image data associated with the vehicle.

8. A method, comprising:

assigning vehicles to support queues based on state data;

generating a map display including locations of the vehicles;

generating a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data; and

tracking autonomous operation of the vehicles.

9. The method of claim 8 , further comprising:

receiving a request from a vehicle for remote support; and

transmitting instruction data to the vehicle to provide the remote support.

10. The method of claim 8 , further comprising:

responsive to autonomous operation of a vehicle operating outside of defined parameter values, transmitting instruction data to the vehicle to modify the autonomous operation of the vehicle.

11. The method of claim 10 , wherein the support queues comprise first level control stations, each first level control station receiving the state data for different vehicles.

12. The method of claim 11 , further comprising:

prior to transmitting the instruction data to the vehicle, assigning, by a second level control station, the vehicle from a first control station to a second control station based on a change in the state data for the vehicle.

13. The method of claim 8 , further comprising:

responsive to an input signal to a first state indicator representing state data of a vehicle, generating a remote support interface display including image data associated with the vehicle.

14. The method of claim 13 , wherein the image data comprises any of an image of an individual within the vehicle and an image of an environment around the vehicle.

15. The method of claim 13 , wherein the image data comprises an image of an obstruction in proximity to the vehicle, the method further comprising:

determining a new route that avoids the obstruction and deviates from a current route; and

transmitting the new route to the vehicle to modify autonomous operation of the vehicle.

16. The method of claim 15 , further comprising:

analyzing the state data of at least one of the vehicles other than the vehicle in proximity to the obstruction; and

transmitting instruction data to the at least one of the vehicles to modify the autonomous operation of the at least one of the vehicles.

17. The method of claim 8 , further comprising:

ranking the vehicles based on the state data; and

generating the state display by assigning each of the vehicles to a respective one of the support queues based on the ranking.

18. The method of claim 8 , further comprising:

assigning at least one vehicle from one of the support queues that is determined as being over-tasked to another of the support queues that is determined as not being over-tasked.

19. The method of claim 8 , wherein the state data includes an issue state, the issue state indicating any of a normal state, an issue with an operator, an issue with a passenger, an issue with traffic, an issue with an external environment, a physical issue of the vehicle, and an issue related to a decision based on autonomous operation of the vehicle.

20. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium includes executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:

assigning vehicles to support queues based on state data;

generating a map display including locations of the vehicles;

generating a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data; and

tracking autonomous operation of the vehicles.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2022
From: NISSAN NORTH AMERICA, INC.
To: NISSAN MOTOR CO., LTD.
Reel/Frame 060656/0554 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2020
From: BUALAT, MARIA
To: UNITED STATES OF AMERICA AS REPRESENTED BY THE ADMINISTRATOR OF NASA
Reel/Frame 052813/0368 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2020
From: FONG, TERRENCE
To: UNITED STATES OF AMERICA AS REPRESENTED BY THE ADMINISTRATOR OF NASA
Reel/Frame 052813/0435 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2020
From: ALLAN, MARK; SCHAFER, ERIC
To: UNITED STATES OF AMERICA AS REPRESENTED BY THE ADMINISTRATOR OF NASA
Reel/Frame 052813/0494 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2020
From: PEDERSEN, LIAM; SIERHUIS, MAARTEN; UTZ, HANS
To: NISSAN NORTH AMERICA, INC.
Reel/Frame 052813/0584 →
Continuity (3)
Continuation 16348726
Provisional Application 62428002 · Nov 30, 2016
Related Publication 20200293065A1 · Sep 17, 2020