IP Library Granted Patent US 11,738,772
Granted Patent B1
US 11,738,772 · App. 17/127,450 · Granted Aug 29, 2023

Object avoidance with perceived safety subgoal

Inventors: Andrew E. Beller (San Francisco, CA); Ari Joseph Goldberg (San Mateo, CA)
Assignee: Zoox, Inc.
B60W60/0011B60W30/09B60W30/0956B60W60/0027B60W10/04B60W10/18B60W10/20B60W2520/10B60W2554/4029B60W2554/801
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,738,772
App. No.
17/127,450
Granted
Aug 29, 2023
Kind
B1
Abstract

Techniques for determining a speed for a vehicle as it traverses an environment with pedestrians are discussed herein. For example, a vehicle computing system may implement techniques to determine an action for a vehicle to take based on a detected pedestrian in an environment. The vehicle computing system may receive sensor data of an environment from a sensor associated with a vehicle, determine, based at least in part on the sensor data, an object in the environment and receive a predicted object trajectory associated with the object. The vehicle computing system may then determine, based on the predicted object trajectory, a distance between a simulated vehicle passing location and a predicted object location, determine, based on the distance, a speed, determine, based on the distance and the speed, a trajectory for the vehicle to follow, and control the vehicle based on the trajectory.

Claims (68)

1. A vehicle system comprising:

a sensor;

one or more processors; and

memory storing processor-executable instructions that, when executed by the one or more processors, configure the vehicle system to:

receive sensor data of an environment from the sensor;

identify a pedestrian at a location in the environment based at least in part on the sensor data;

determine a predicted trajectory associated with the pedestrian;

determine, based on the predicted trajectory, that the pedestrian will be within a threshold distance of the vehicle as the vehicle operates according to a first vehicle trajectory associated with the vehicle;

determine a second vehicle trajectory based at least in part on the predicted trajectory of the pedestrian, wherein the second vehicle trajectory is associated with the vehicle navigating to avoid the pedestrian, wherein the vehicle is configured to determine the second vehicle trajectory by:

determining a physical characteristic of a pedestrian;

determining, based on a perceived safe distance metric and the physical characteristic, a lateral distance associated with navigating to avoid the pedestrian; and

determining, based on a perceived safe speed metric and the physical characteristic, a vehicle speed associated with navigating to avoid the pedestrian based at least in part on the lateral distance;

determining the second vehicle trajectory based at least in part on the lateral distance and the vehicle speed; and

controlling the vehicle based on the second vehicle trajectory.

2. The vehicle system as claim 1 recites, wherein determining the vehicle speed is based at least in part on a perceived safe speed cost and the perceived safe speed cost based on a perceived safe speed, the perceived safe speed being determined based on the lateral distance by determining the perceived safe speed as a result of a mathematical function using the lateral distance, the mathematical function being determined based on previous lateral distances and associated speeds in passing interactions of vehicles and objects.

3. The vehicle system as claim 1 recites, wherein the instructions further configure the vehicle to determine the second vehicle trajectory by:

determining, based on the predicted trajectory, a distance to the pedestrian; and

wherein determining the second vehicle trajectory comprises determining a cost optimization function based at least in part on a weighting of a perceived safe speed cost, the weighting based at least in part on the distance to the pedestrian, the weighting increasing as the distance remaining to the pedestrian decreases.

4. The vehicle system as claim 1 recites, wherein the determining the vehicle speed is based at least in part on a perceived safe speed cost and the determining the second vehicle trajectory comprises a cost optimization function including a weighting of the perceived safe speed cost, the weighting based at least in part on the lateral distance, the weighting increasing as the lateral distance decreases.

5. The vehicle system as claim 1 recites, wherein the determining the vehicle speed is based at least in part on a perceived safe speed cost and the perceived safe speed cost is based on a difference between the vehicle speed and a perceived safe speed associated with the lateral distance.

6. A method comprising:

receiving sensor data of an environment from a sensor associated with a vehicle;

determining, based at least in part on the sensor data, an object in the environment;

receiving a predicted object trajectory associated with the object;

determining, based on the predicted object trajectory, a predicted object location;

determining a physical characteristic of the object, wherein the object is a pedestrian;

determining, based on the predicted object location, the predicted object trajectory, and the physical characteristic, a distance between a predicted vehicle passing location and the predicted object location or a speed of the vehicle in proximity to the predicted object location;

determining, based a perceived safety metric, and the distance or the speed, a trajectory for the vehicle to follow; and

controlling the vehicle based on the trajectory.

7. The method as claim 6 recites, wherein determining the speed comprises:

determining a perceived safe speed based on the distance comprising:

retrieving the perceived safe speed from a look-up table of a plurality of perceived safe speeds indexed based on distance; or

determining the perceived safe speed based on a result of a mathematical function based on the distance.

8. The method of claim 7 , wherein determining the trajectory for the vehicle to follow comprises excluding one or more possible trajectories that do not include the vehicle traveling at one of the plurality of perceived safe speeds associated with a plurality of corresponding distances between corresponding simulated vehicle passing locations and predicted object locations.

9. The method of claim 7 , wherein the distance is a lateral distance associated with navigating around the object; and

the speed is a vehicle speed associated with navigating around the object based at least in part on a perceived safe speed cost determined based at least in part on the lateral distance.

10. The method of claim 9 , wherein the perceived safe speed cost is based on the perceived safe speed, the perceived safe speed being determined based on the lateral distance by determining the perceived safe speed as the result of the mathematical function based on the lateral distance, the mathematical function being determined based on previous lateral distances and associated speeds of passing interactions of vehicles and objects.

11. The method of claim 6 , wherein determining the trajectory further comprises:

determining a different speed based on the distance;

determining a second cost based on the distance and the different speed is less than a first cost based on the distance and the speed; and

determining, based on the determining the second cost is less than the first cost, the trajectory based on the distance and the different speed.

12. The method of claim 6 , wherein determining the speed further comprises:

determining a classification of the predicted object location; and

determining the speed based at least in part on the distance and the classification of the predicted object location.

13. The method of claim 6 , wherein the physical characteristic of the pedestrian comprises

one of an arm length or a height of the pedestrian.

14. One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations comprising:

receiving sensor data of an environment from a sensor associated with a vehicle;

determining, based at least in part on the sensor data, an object in the environment;

receiving a predicted object trajectory associated with the object;

determining, based on the predicted object trajectory, a predicted object location;

determining a physical characteristic of the predicted object, wherein the predicted object is a pedestrian;

determining, based on the predicted object location, the predicted object trajectory, and the physical characteristic, a distance between a predicted vehicle passing location and the predicted object location or a speed of the vehicle in proximity to the predicted object location;

determining, based a perceived safety metric, and the distance or the speed, a trajectory for the vehicle to follow; and

controlling the vehicle based on the trajectory.

15. The one or more non-transitory computer-readable media of claim 14 , wherein determining the speed comprises:

determining a perceived safe speed based on the distance comprising:

retrieving the perceived safe speed from a look-up table of a plurality of perceived safe speeds indexed based on distance; or

determining the perceived safe speed based on a result of a mathematical function based on the distance.

16. The one or more non-transitory computer-readable media of claim 15 , wherein determining the trajectory for the vehicle to follow comprises excluding one or more possible trajectories that do not include the vehicle traveling at one of a plurality of perceived safe speeds associated with a plurality of corresponding distances between corresponding simulated vehicle passing locations and predicted object locations.

17. The one or more non-transitory computer-readable media of claim 15 , wherein:

the distance is a lateral distance associated with navigating around the object; and

the speed is a vehicle speed associated with navigating around the object based at least in part on a perceived safe speed cost determined based at least in part on the lateral distance.

18. The one or more non-transitory computer-readable media of claim 17 , wherein the perceived safe speed cost is based on the perceived safe speed, the perceived safe speed being determined based on the lateral distance by determining the perceived safe speed as the result of the mathematical function based on the lateral distance, the mathematical function being determined based on previous lateral distances and associated speeds of passing interactions of vehicles and objects.

19. The one or more non-transitory computer-readable media of claim 18 , wherein the operations further comprise:

determining a distance to the object; and

wherein determining the trajectory comprises determining a cost optimization function based at least in part on a weighting of the perceived safe speed cost, the weighting based at least in part on the distance to the object, the weighting increasing as the distance remaining to the object decreases.

20. The one or more non-transitory computer-readable media of claim 18 , wherein determining the trajectory comprises determining a cost optimization function based at least in part on a weighting of the perceived safe speed cost, the weighting based at least in part on the lateral distance, the weighting increasing as the lateral distance decreases.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 11, 2021
From: BELLER, ANDREW E.; GOLDBERG, ARI JOSEPH
To: ZOOX, INC.
Reel/Frame 057146/0620 →
Cited By (6)
US 12,197,222 US 12,223,677 US 12,280,797 US 12,441,318 US 12,485,909 US 12,497,032