IP Library Granted Patent US 11,181,905
Granted Patent B2
US 11,181,905 · App. 16/530,837 · Granted Nov 23, 2021

Teleoperation of autonomous vehicles

Inventors: Ayman Alalao (Cambridge, MA); Hsun-Hsien Chang (Brookline, MA); Shih-Yuan Liu (Boston, MA)
Assignee: MOTIONAL AD LLC
G05D1/0022G05D1/0055G05D1/0088G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,181,905
App. No.
16/530,837
Granted
Nov 23, 2021
Kind
B2
Abstract

An autonomous vehicle receives sensor data from one or more sensors of the autonomous vehicle, and generates a request for remote control of the autonomous vehicle by a computer system remote from the autonomous vehicle. Generating the request includes determining a quality metric associated with a network connection between the autonomous vehicle and the computer system, and upon determining that the quality metric is greater than a threshold quality level, transmitting, from the autonomous vehicle to the computer system the request for remote control and a first data item representing the sensor data. The first data item meets one or more conditions associated with the threshold quality level.

Claims (52)

1. A method comprising:

receiving, by an autonomous vehicle, sensor data from one or more sensors of the autonomous vehicle;

generating, by the autonomous vehicle, a request for remote control of the autonomous vehicle by a computer system remote from the autonomous vehicle, wherein generating the request comprises:

determining, by the autonomous vehicle, a quality metric associated with a network connection between the autonomous vehicle and the computer system, and

comparing the quality metric to a threshold quality level,

transmitting, from the autonomous vehicle to the computer system, the request for remote control, wherein transmitting the request comprises:

transmitting, with the request, a first data item representing the sensor data upon determining that the quality metric is greater than threshold quality level, or

transmitting, with the request, a second data item representing the sensor data upon determining that the quality metric is less than threshold quality level, the second data item having a smaller data size or a lower complexity than the first data item,

wherein the first data item comprises at least one video or image of an environment of the autonomous vehicle,

wherein the second data item comprises textual data describing the environment of the autonomous vehicle instead of the at least one video or image.

2. The method of claim 1 , wherein the sensor data comprises at least one of a video, an image, or proximity data captured by the one or more sensors.

3. The method of claim 1 , wherein the quality metric corresponds to an available bandwidth of the network connection.

4. The method of claim 1 , wherein the quality metric corresponds to a latency associated with network connection.

5. The method of claim 1 , wherein the quality metric corresponds to a reliability of the network connection.

6. The method of claim 1 , wherein determining the quality metric associated with the network connection comprises:

transmitting a beacon signal using the network connection, and

determining the quality metric based on the transmission of the beacon signal.

7. The method of claim 6 , wherein determining the quality metric based on the transmission of the beacon signal comprises determining an available bandwidth of the network connection based on the transmission of the beacon signal.

8. The method of claim 1 , wherein the first data item comprises a portion of sensor data having a higher spatial resolution than that of a portion of sensor data included in the second data item.

9. The method of claim 1 , wherein the first data item comprises a portion of sensor data having a high temporal resolution than that of a portion of sensor data included in the second data item.

10. The method of claim 1 , wherein the textual data comprises one or more data items in a JavaScript Object Notation (JSON), Hypertext Markup Language (HTML), or Extensible Markup Language (XML) format.

11. The method of claim 1 , further comprising:

determining, by the autonomous vehicle, a condition impeding a travel of the autonomous vehicle, and

wherein the request for remote control is generated responsive to determining the condition impeding the travel of the autonomous vehicle.

12. The method of claim 11 , wherein the condition impeding the travel of the autonomous vehicle corresponds to an object blocking a path of the autonomous vehicle.

13. The method of claim 11 , wherein the condition impeding the travel of the autonomous vehicle corresponds to a closure of a roadway along a path of the autonomous vehicle.

14. The method of claim 1 , further comprising:

receiving, at the autonomous vehicle from the computer system, a command signal comprising instructions to the autonomous vehicle to navigate a specified path, and

executing, at the autonomous vehicle, the command signals to navigate the specified path.

15. A device comprising:

one or more processors; and

one or more storage devices storing instructions for execution by the one or more processors,

wherein the one or more processors are configured to:

receive sensor data from one or more sensors of an autonomous vehicle;

generate a request for remote control of the autonomous vehicle by a computer system remote from the autonomous vehicle, wherein generating the request comprises:

determining a quality metric associated with a network connection between the autonomous vehicle and the computer system, and

comparing the quality metric to a threshold quality level, transmit, to the computer system, the request for remote control, wherein

transmitting the request comprises:

transmitting, with the request, a first data item representing the sensor data upon determining that the quality metric is greater than threshold quality level, or

transmitting, with the request, a second data item representing the sensor data upon determining that the quality metric is less than threshold quality level, the second data item having a smaller data size or a lower complexity than the first data item,

wherein the first data item comprises at least one video or image of an environment of the autonomous vehicle,

wherein the second data item comprises textual data describing the environment of the autonomous vehicle instead of the at least one video or image.

16. A non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors the one or more programs including instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving sensor data from one or more sensors of an autonomous vehicle;

generating a request for remote control of the autonomous vehicle by a computer system remote from the autonomous vehicle, wherein generating the request comprises:

determining a quality metric associated with a network connection between the autonomous vehicle and the computer system, and

comparing the quality metric to a threshold quality level,

transmitting, to the computer system, the request for remote control, wherein transmitting the request comprises:

transmitting, with the request, a first data item representing the sensor data upon determining that the quality metric is greater than threshold quality level, or

transmitting, with the request, a second data item representing the sensor data upon determining that the quality metric is less than threshold quality level, the second data item having a smaller data size or a lower complexity than the first data item,

wherein the first data item comprises at least one video or image of an environment of the autonomous vehicle,

wherein the second data item comprises textual data describing the environment of the autonomous vehicle instead of the at least one video or image.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2020
From: APTIV TECHNOLOGIES LIMITED
To: MOTIONAL AD LLC
Reel/Frame 053863/0399 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2020
From: ALALAO, AYMAN; CHANG, HSUN-HSIEN; LIU, SHIH-YUAN
To: APTIV TECHNOLOGIES LIMITED
Reel/Frame 052023/0192 →