IP Library › Granted Patent US 12,617,428
Granted Patent B2
US 12,617,428 · App. 18/204,347 · Granted May 5, 2026

Action reference generation pipeline

Inventors: Liam Gallagher (San Francisco, CA); Matthew Van Heukelom (San Francisco, CA)
Assignee: Zoox, Inc.
B60W60/0011B60W30/09B60W30/0956B60W2554/20B60W2554/80
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,617,428
App. No.
18/204,347
Granted
May 5, 2026
Kind
B2
Abstract

Techniques for accurately predicting and avoiding collisions with objects detected in an environment of a vehicle are discussed herein. A vehicle computing device can implement a model to generate action references usable in a tree search to control the vehicle. For example, the model may represent one or more trajectory generators that modify a baseline trajectory in different ways to output a set of actions trajectories that are usable to define a tree structure. The set of action trajectories can be used by the vehicle computing device for predicting vehicle actions by the vehicle computing device to control the vehicle.

Claims (77)

1 . A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:

receiving a baseline trajectory associated with a vehicle in an environment, the baseline trajectory including a direction, a velocity, and an acceleration;

inputting the baseline trajectory into a generator to modify one or more points along the baseline trajectory;

determining, based at least in part on an output from the generator, a set of action trajectories associated with the vehicle;

defining a portion of branches of a tree structure based at least in part on the set of action trajectories, the tree structure comprising a same start point for the set of action trajectories;

determining, based at least in part on the portion of branches of the tree structure, a candidate trajectory; and

controlling the vehicle based at least in part on the candidate trajectory.

2 . The system of claim 1 , the operations further comprising:

modifying, based at least in part on a control policy, a shape of at least one action trajectory in the set of action trajectories to output a modified set of action trajectories,

wherein the tree structure is further defined based at least in part on the modified set of action trajectories.

3 . The system of claim 1 , wherein the output is a first output, and the operations further comprising:

determining, by the generator, presence of a static object in the environment; and

receiving, as a second output from the generator, an action trajectory in the set of action trajectories to avoid the static object.

4 . The system of claim 1 , wherein an action trajectory in the set of action trajectories is defined by multiple points, the generator is a first generator, the output is a first output, and the operations further comprising:

inputting the action trajectory into a second generator configured to one or more of:

relocating one of the multiple points along the action trajectory from a first location to a second location along the action trajectory,

modifying a vehicle state associated with one of the multiple points of the action trajectory,

modifying a number of the multiple points of the action trajectory,

increasing a length of the action trajectory, or

determining a stopping action associated with the action trajectory; and

receiving a second output from the second generator,

wherein the set of action trajectories comprises the second output.

5 . The system of claim 1 , wherein at least one of the set of action trajectories defines a node of the tree structure.

6 . One or more non-transitory computer readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:

receiving a trajectory associated with a vehicle;

inputting the trajectory into a component;

receiving, from the component, a set of action trajectories comprising a first action trajectory that includes a first modification of the trajectory and a second action trajectory that includes a second modification of the trajectory;

inputting the set of action trajectories and the trajectory into a tree search algorithm that searches the first action trajectory, the second action trajectory, and the trajectory in a tree structure, wherein the first action trajectory and the second action trajectory start at a same root node of the tree structure;

receiving, as an output of the tree search algorithm, a candidate trajectory; and

controlling the vehicle based at least in part on the candidate trajectory.

7 . The one or more non-transitory computer readable media of claim 6 , wherein the component determines a first trajectory of the set of action trajectories based at least in part on modifying, based at least in part on a control policy, a shape of at least the trajectory.

8 . The one or more non-transitory computer readable media of claim 6 , wherein the component is a first component, and the operations further comprise:

inputting the trajectory to a second component;

determining, by the second component, presence of a static object in an environment; and

receiving, from the second component, a second trajectory of the set of action trajectories configured to control the vehicle to avoid the static object.

9 . The one or more non-transitory computer readable media of claim 6 , wherein the component is a first component, the output is a first output, and the operations further comprising:

inputting the set of action trajectories to a second component;

performing, by the second component, one or more of:

relocating one of multiple points along an action trajectory from a first location to a second location between a first end and a second end of the action trajectory,

modifying a number of the multiple points of the action trajectory,

extending at least one of the multiple points from the second end of the action trajectory to a position in an environment, or

determining a stopping action associated with the action trajectory; and

receiving a second output from the second component,

wherein the set of action trajectories is determined based at least in part on the second output from the second component.

10 . The one or more non-transitory computer readable media of claim 6 , wherein the output is a first output and a second output from the component is an input to another component.

11 . The one or more non-transitory computer readable media of claim 6 , wherein inputting the set of action trajectories into the tree search algorithm comprises:

defining the tree structure to include a branch for at least one action trajectory in the set of action trajectories.

12 . The one or more non-transitory computer readable media of claim 6 , wherein:

determining the set of action trajectories comprises the component performing a first transformation using a first model and a second transformation using a second model different from the first model.

13 . The one or more non-transitory computer readable media of claim 6 , the operations further comprising:

transmitting the set of action trajectories associated with the tree search algorithm to a vehicle computing device;

determining, based at least in part on a search of the tree structure, the candidate trajectory for the vehicle to use at a future time; and

controlling the vehicle in an environment using the candidate trajectory.

14 . The one or more non-transitory computer readable media of claim 6 , the operations further comprising:

determining an identifier associated with the trajectory;

determining information describing a data structure of the trajectory; and

storing the identifier and the information as profile data in a storage device.

15 . The one or more non-transitory computer readable media of claim 6 , the operations further comprising:

determining metadata describing an action trajectory in the set of action trajectories, the metadata including a source of the action trajectory and information of a segment of the action trajectory; and

associating the metadata with the tree search algorithm.

16 . The one or more non-transitory computer readable media of claim 6 , where the output is a first output and the operations further comprising:

determining the set of action trajectories based at least in part on a second output by a parallel processing unit.

17 . A method comprising:

receiving a trajectory associated with a vehicle;

inputting the trajectory into a component;

receiving, from the component, a set of action trajectories comprising a first action trajectory that includes a first modification of the trajectory and a second action trajectory that includes a second modification of the trajectory;

inputting the set of action trajectories and the trajectory into a tree search algorithm that searches the first action trajectory, the second action trajectory, and the trajectory in a tree structure, wherein the first action trajectory and the second action trajectory comprise a same start point in the tree structure;

receiving, as an output of the tree search algorithm, a candidate trajectory; and

controlling the vehicle based at least in part on the candidate trajectory.

18 . The method of claim 17 , wherein the component determines the first action trajectory of the set of action trajectories based at least in part on modifying, based at least in part on a control policy, a shape of at least the trajectory.

19 . The method of claim 17 , wherein the component is a first component, and further comprising:

inputting the trajectory to a second component;

determining, by the second component, presence of a static object in an environment; and

receiving, from the second component, a second trajectory of the set of action trajectories configured to control the vehicle to avoid the static object.

20 . The method of claim 17 , wherein the output is a first output and a second output from the component is an input to another component.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 5, 2023
From: GALLAGHER, LIAM; VAN HEUKELOM, MATTHEW
To: ZOOX, INC.
Reel/Frame 063858/0569 →
Continuity (1)
Related Publication 20240400096A1 · Dec 5, 2024
References Cited (10)
US 9568915B1 · Berntorp · 2017 [cited by examiner]
US 20220194419A1 · Houshmand et al. · 2022 [cited by applicant]
US 20230041975A1 · Caldwell et al. · 2023 [cited by applicant]
US 20230097121A1 · Schwartz et al. · 2023 [cited by applicant]
US 20240059285A1 · Ng · 2024 [cited by examiner]
EP 2169500A1 · 2010 [cited by applicant]
JP 2009281942A · 2009 [cited by applicant]
C. Paxton, V. Raman, G. D. Hager and M. Kobilarov, “Combining neural networks and tree search for task and motion planning in challenging environments,” 2017 IEEE/RSJ International Conference on Intelligent Robots and S… [cited by examiner]
Fulgenzi, Chiara, et al. “Probabilistic navigation in dynamic environment using rapidly-exploring random trees and gaussian processes.” 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 200… [cited by examiner]
Search Report and Written Opinion for International Application No. PCT/US2024/029326, Dated Sep. 3, 2024, 12 pages. [cited by applicant]