IP Library › Granted Patent US 11,787,438
Granted Patent B2
US 11,787,438 · App. 17/125,890 · Granted Oct 17, 2023

Collaborative vehicle path generation

Inventors: Arian Houshmand (San Francisco, CA); Ravi Verma Gogna (San Jose, CA); Mark Jonathon McClelland (San Francisco, CA)
Assignee: Zoox, Inc.
B60W60/0011B60W60/0023G01C21/3469G05D1/0038G05D1/0088B60W2556/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,787,438
App. No.
17/125,890
Granted
Oct 17, 2023
Kind
B2
Abstract

A teleoperations system that collaboratively works with an autonomous vehicle guidance system to generate a path for controlling the autonomous vehicle may comprise generating one or more trajectories at the teleoperations system based at least in part on environment data received from the autonomous vehicle and presenting the one or more trajectories to a teleoperator (e.g., a human user, machine-learned model, or artificial intelligence component). A selection of one of the trajectories may be received at the teleoperations system and transmitted to the autonomous vehicle. The one or more trajectories may be generated at the teleoperations system and/or received from the autonomous vehicle. Regardless, the autonomous vehicle may generate a control trajectory based on the trajectory received from teleoperations, instead of merely implementing the trajectory from the teleoperations system.

Claims (65)

1. A method comprising:

receiving, at a device remote from an autonomous vehicle, a request for assistance and sensor data associated with an environment of the autonomous vehicle;

determining, based at least in part on the sensor data, two or more candidate trajectories, wherein the two or more candidate trajectories comprise separate instructions configured to actuate drive components of the autonomous vehicle;

displaying, at the device, a representation of the environment and the two or more candidate trajectories;

receiving, at the device, input identifying a proposed trajectory from among the two or more candidate trajectories; and

transmitting the proposed trajectory to the autonomous vehicle, the autonomous vehicle configured to determine a path for controlling the autonomous vehicle based at least in part on the proposed trajectory, wherein the path comprises a first control trajectory that the autonomous vehicle determines based at least in part on the proposed trajectory and a second control trajectory that is based at least in part on the first control trajectory.

2. The method of claim 1 , wherein:

transmitting the proposed trajectory to the autonomous vehicle comprises transmitting a weight associated with the proposed trajectory; and

the autonomous vehicle generates a set of candidate trajectories based at least in part on the sensor data and the weight and determines one of the set of candidate trajectories based at least in part on the weight.

3. The method of claim 1 , further comprising:

determining a second set of candidate trajectories based at least in part on the proposed trajectory, the second set of candidate trajectories comprising a start position associated with an end position of the proposed trajectory;

receiving input identifying a second proposed trajectory from among the second set of candidate trajectories; and

transmitting the second proposed trajectory to the autonomous vehicle, wherein the path is further based at least in part on the second proposed trajectory.

4. The method of claim 1 , further comprising:

receiving, from the autonomous vehicle, a second set of candidate trajectories generated by the autonomous vehicle based at least in part on the proposed trajectory;

receiving input identifying a second proposed trajectory from among the second set of candidate trajectories; and

transmitting the second proposed trajectory to the autonomous vehicle, wherein the path is further based at least in part on the second proposed trajectory.

5. The method of claim 1 , wherein determining the two or more candidate trajectories is based at least in part on at least one of object data or map data determined based at least in part on the sensor data.

6. The method of claim 1 , wherein the proposed trajectory identifies at least one of a region, a lane, a location, or a weight that is used as part of a search algorithm running on the autonomous vehicle to identify the path.

7. A system comprising:

one or more processors; and

a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:

receiving, at a device remote from an autonomous vehicle, a request for assistance, a trajectory for the autonomous vehicle, and data associated with an environment of the autonomous vehicle;

determining, at the device, based on the data and the trajectory, in response to receiving the request for assistance, a proposed trajectory, wherein the proposed trajectory comprises instructions for actuating a drive component of the autonomous vehicle; and

transmitting the proposed trajectory to the autonomous vehicle, wherein the autonomous vehicle is configured to determine a control trajectory based at least in part on the proposed trajectory.

8. The system of claim 7 , wherein the operations further comprise:

determining, by the one or more processors, a second set of candidate trajectories based at least in part on the proposed trajectory, the second set of candidate trajectories comprising a start position associated with an end position of the proposed trajectory; and

transmitting a second proposed trajectory to the autonomous vehicle, the second proposed trajectory being determined from among the second set of candidate trajectories.

9. The system of claim 7 , wherein the operations further comprise:

receiving, from the autonomous vehicle, a second set of candidate trajectories generated by the autonomous vehicle based at least in part on the proposed trajectory;

identifying a second proposed trajectory from among the second set of candidate trajectories; and

transmitting the second proposed trajectory to the autonomous vehicle.

10. The system of claim 7 , wherein the operations further comprise:

generating a plurality of proposed trajectories for the autonomous vehicle including the proposed trajectory;

wherein the proposed trajectory is determined, from among the plurality of proposed trajectories, based on a weighting associated with the trajectory of the autonomous vehicle received at the device.

11. The system of claim 7 , wherein the proposed trajectory is determined based at least in part on:

historic data associated with a vehicle in a similar environment as the autonomous vehicle; or

additional data that provides addition information regarding the environment than the data associated with the environment received from the autonomous vehicle.

12. One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving first sensor data from one or more sensors associated with an autonomous vehicle and a target location;

determining, based at least in part on the first sensor data and the target location, a first trajectory;

determining a confidence score associated with the first trajectory is less than a threshold;

transmitting, based at least in part on determining that the confidence score is less than the threshold, to a remote computing device, a request for teleoperator input;

receiving, from the remote computing device, a proposed trajectory comprising first instructions configured to actuate drive components of the autonomous vehicle;

receiving second sensor data from the one or more sensors associated with the autonomous vehicle;

determining, based at least in part on the second sensor data, a second trajectory; and

determining a control trajectory based at least in part on the proposed trajectory and the second trajectory, the control trajectory comprising second instructions configured to actuate the drive components.

13. The one or more non-transitory computer-readable media of claim 12 , wherein:

the operations further comprise receiving a weight associated with the proposed trajectory; and

determining the control trajectory comprises:

determining a cost associated with the control trajectory based at least in part on the weight; and

determining the control trajectory from a set of trajectories based at least in part on the cost.

14. The one or more non-transitory computer-readable media of claim 13 , wherein determining the control trajectory is based at least in part on a tree search.

15. The one or more non-transitory computer-readable media of claim 12 , wherein the operations further comprise:

transmitting a set of trajectories from the autonomous vehicle to the remote computing device, wherein the request for teleoperator input comprises the set of trajectories generated by the autonomous vehicle; and

wherein the proposed trajectory is one of the set of trajectories sent to the remote computing device.

16. The one or more non-transitory computer-readable media of claim 12 , wherein the operations further comprise:

at least one of:

receiving, from the remote computing device, a third trajectory; or

determining the second trajectory based at least in part on the control trajectory,

wherein the third trajectory comprises a start position associated with an end position of the control trajectory.

17. The one or more non-transitory computer-readable media of claim 12 , wherein the first trajectory is different from the second trajectory.

18. The one or more non-transitory computer-readable media of claim 12 , wherein the control trajectory is different than the proposed trajectory.

19. The one or more non-transitory computer-readable media of claim 12 , wherein the proposed trajectory identifies at least one of a region, a lane, a location, or a weight that is used as part of a search algorithm executed by the autonomous vehicle to identify the control trajectory.

20. The one or more non-transitory computer-readable media of claim 12 , wherein the control trajectory is generated at the remote computing device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2021
From: HOUSHMAND, ARIAN; GOGNA, RAVI; MCCLELLAND, MARK JONATHON
To: ZOOX, INC.
Reel/Frame 057007/0810 →
Continuity (1)
Related Publication 20220194419A1 · Jun 23, 2022
Cited By (3)
US 12,311,970 US 12,311,984 US 12,443,182