IP Library › Granted Patent US 11,442,460
Granted Patent B2
US 11,442,460 · App. 16/715,736 · Granted Sep 13, 2022

External control of an autonomous vehicle

Inventors: Marcus Hammond (Mountain View, CA); Marc Wimmershoff (Palo Alto, CA); Timothy David Kentley-Klay (Stanford, CA)
Assignee: Zoox, Inc.
G05D1/0223B60W30/09G05D1/0011G05D1/024G05D1/0214G05D1/0276B60W2520/10B60W2554/00G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,442,460
App. No.
16/715,736
Granted
Sep 13, 2022
Kind
B2
Abstract

Remote controlling of a vehicle, such as an autonomous vehicle, may sometimes be more efficient and/or reliable. Such control, however, may require processes for ensuring safety of surrounding persons and objects. Aspects of this disclosure include using onboard sensors to detect objects in an environment and alter remote commands according to such objects, e.g. by reducing a maximum permitted velocity of the vehicle as a function of distance to detected objects. In some examples described herein, such remote controlling may be performed by using objects in the environment as control objects, with movements of the control objects resulting in movement of the vehicle.

Claims (89)

1. A vehicle selectively configurable for remote operation comprising:

a sensor;

a drive system configured to cause the vehicle to move through an environment;

one or more processors; and

one or more non-transitory computer readable media having instructions stored thereon that when executed by the one or more processors cause the one or more processors to perform operations comprising:

configuring the vehicle in a first configuration for autonomous control;

controlling the vehicle autonomously in the first configuration;

receiving, with the vehicle in the first configuration, an instruction to remotely control the vehicle;

authenticating a remote controller, the remote controller being remote from the vehicle and configured to generate control commands for controlling the vehicle;

configuring, in response to the authenticating, the vehicle in a second configuration for remote control;

receiving, from the remote controller and with the vehicle in the second configuration, a control command of the control commands comprising a command for the vehicle to proceed in a direction;

receiving sensor data from the sensor comprising information about an object in the environment;

determining, from the sensor data, a distance of the object from the vehicle;

determining, based at least in part on the control command and the distance, a modified control command; and

controlling the one or more drive systems to move the vehicle in accordance with the modified control command.

2. The vehicle of claim 1 , wherein the remote controller is paired via a short field wireless protocol.

3. The vehicle of claim 1 , the operations further comprising pairing the remote controller with the vehicle, wherein the pairing comprises communicating with a receiver selectively physically coupled to the vehicle and in wireless communication with the remote controller.

4. The vehicle of claim 1 , wherein the control system is further configured to:

determine a first safety region extending a first distance in the direction; and

determine a second safety region extending a second distance in the direction, wherein the second distance is greater than the first distance,

wherein the modified control command is determined based at least in part on the object being located in at least one of the first safety region or the second safety region.

5. The vehicle of claim 4 , wherein:

the command further comprises a commanded speed,

the modified control command is based at least in part on a modified speed for the vehicle, and

the modified speed is zero when the distance indicates that the object is in the first safety region.

6. The vehicle of claim 4 , wherein:

the command comprises a commanded speed,

the modified control command is based at least in part on a modified speed for the vehicle, and

the modified speed is a non-zero speed that is less than the commanded speed when the distance indicates that the object is in the second safety region.

7. The vehicle of claim 1 , wherein:

the command comprises a commanded speed; and

the commanded speed is equal to or below about 10 km/h.

8. The vehicle of claim 1 , wherein the control system is further configured to authenticate the remote controller based at least in part on one or more of:

establishing a connection with the remote controller;

synchronizing the remote controller with the vehicle; or

recognizing the remote controller using at least one of pattern recognition or code recognition.

9. A method of selectively remotely controlling a vehicle comprising:

configuring the vehicle in a first configuration for autonomous control;

controlling the vehicle in the first configuration;

receiving an instruction to remotely control the vehicle;

authenticating, at the vehicle, a remote controller remote from the vehicle, the remote controller being configured to generate control commands for controlling the vehicle;

reconfiguring the vehicle in a second configuration for remote control;

receiving, from the remote controller and with the vehicle in the second configuration, a control command of the control commands for the vehicle to proceed in an environment;

receiving, from a sensor disposed on the vehicle, sensor data corresponding to the environment;

determining, from the sensor data, object information about an object in the environment;

based at least in part on the object information, generating a modified control command; and

controlling the vehicle to move in accordance with the modified control command.

10. The method of claim 9 , wherein pairing the remote controller comprises at least one of establishing a wireless connection with the remote controller or establishing communication with a receiver selectively physically coupled to the vehicle, the receiver being configured to receive the control command from the remote controller.

11. The method of claim 10 , wherein the authenticating the remote controller is based at least in part on one or more of:

establishing a connection with the remote controller;

synchronizing the remote controller with the vehicle; or

recognizing the remote controller using at least one of pattern recognition or code recognition.

12. The method of claim 9 , further comprising:

determining a first safety region extending a first distance in a direction of travel of the vehicle;

determining a second safety region extending from the first safety region a second distance in the direction; and

determining, based on the object information, an object position of the object relative to the vehicle,

wherein the modified control command is determined based at least in part on the object position indicating that the object is located in at least one of the first safety region or the second safety region.

13. The method of claim 9 , further comprising:

determining a safety region extending a distance lateral to a direction of travel of the vehicle; and

determining, based on the object information, an object position of the object relative to the vehicle,

wherein the modified control command is determined based at least in part on the object position indicating that the object is located in the safety region.

14. One or more computer readable media storing instructions that, when executed by one or more processors, cause a control system of a vehicle to perform acts comprising:

operating the vehicle in a first configuration for autonomous control;

receiving, with the vehicle in the autonomous configuration, an instruction to remotely control the vehicle;

authenticating, at the vehicle, a remote controller remote from the vehicle, the remote controller being configured to generate control commands for controlling the vehicle;

reconfiguring the vehicle to a second configuration for remote control;

receiving, from the remote controller and with the vehicle in the second configuration, a control command of the control commands for the vehicle to move in a direction in an environment;

receiving, from a sensor associated with the vehicle, sensor data corresponding to the environment;

determining, from the sensor data, a distance of an object from the vehicle;

generating, based at least in part on the distance, a modified control command; and

generating instructions based at least in part on the modified control command.

15. The one or more computer readable media of claim 14 , wherein:

the control command includes a commanded speed; and

the modified control command comprises a modified speed less than the commanded speed.

16. The one or more computer readable media of claim 15 , wherein:

the modified speed is zero when the distance is less than or equal to a first threshold distance; and

the modified speed is a non-zero speed when the distance is between the first threshold distance and a second threshold difference greater than the first threshold distance.

17. The one or more computer readable media of claim 15 , wherein the modified speed is determined as a linear function of the distance.

18. The one or more computer readable media of claim 14 , the acts further comprising:

determining a safety region extending a distance lateral to a direction of travel of the vehicle; and

determining, based on the object information, an object position of the object relative to the vehicle,

wherein the modified control command is determined based at least in part on the object position indicating that the object is located in the safety region.

19. The one or more computer readable media of claim 14 , the acts further comprising:

pairing the remote controller,

wherein the pairing the remote controller comprises at least one of establishing a wireless connection with the remote controller or establishing communication with a receiver selectively physically coupled to the autonomous vehicle, the receiver being configured to receive the control command from the remote controller.

20. The one or more computer readable media of claim 19 , the authenticating the remote controller being based at least in part on one or more of:

establishing a connection with the remote controller;

synchronizing the remote controller with the autonomous vehicle; or

recognizing the remote controller using at least one of pattern recognition or code recognition.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 12, 2020
From: HAMMOND, MARCUS; WIMMERSHOFF, MARC; KENTLEY-KLAY, TIMOTHY DAVID
To: ZOOX, INC.
Reel/Frame 052640/0676 →
Continuity (2)
Continuation 15833695 · Dec 6, 2017
Related Publication 20200192394A1 · Jun 18, 2020