Granted Patent B1

US 12,649,462 · App. 17/900,332 · Granted Jun 9, 2026

Inventors: Yan Chang (Mountain View, CA); Gowtham Garimella (Hayward, CA); Marin Kobilarov (Baltimore, MD); Gary Linscott (Seattle, WA)

Assignee: Zoox, Inc.

B60W30/0956B60W30/12B60W30/18163B60W50/00B60W2050/0028B60W2555/60

Abstract

Claims (104)

1 . A vehicle comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations comprising:

receiving route data associated with a start position and an end position in an environment;

generating a tree structure by:

associating a current state of the vehicle with a first node of the tree structure;

inputting, into a machine learned model, a representation of the environment associated with the first node;

generating, using the machine learned model, a first candidate action for controlling motion of the vehicle relative to the first node;

determining, based at least in part on a heuristic, a second candidate action for controlling motion of the vehicle relative to the first node, wherein the second candidate action is distinct from the first candidate action;

creating a set of connected nodes of the tree structure based at least in part on the first candidate action and the second candidate action, wherein the first node is a parent node and the first and second candidate actions correspond to child nodes associated with the first node;

determining, using the tree structure, a control trajectory based at least in part on a cost associated with the set of connected nodes in the tree structure relative to the first node having a minimum cost; and

controlling the vehicle in the environment, based at least in part on the control trajectory.

2 . The vehicle of claim 1 , wherein the first or second candidate action comprises one or more of:

a follow action,

a stay in lane action,

a change lane action, or

a turn action.

3 . The vehicle of claim 1 , wherein the cost is associated with one or more of:

a safety cost,

a progress cost,

a comfort cost,

an energy efficiency cost, or

a law abidance cost.

4 . The vehicle of claim 1 , wherein the machine learned model comprises a first machine learned model and the representation of the environment comprises an embedding, the operations further comprising:

inputting, into a second machine learned model, the embedding; and

receiving, from the second machine learned model, a predicted object trajectory associated with an object in the environment, and

wherein the first candidate action is based at least in part on the predicted object trajectory.

5 . The vehicle of claim 1 , wherein generating the tree structure further comprises:

selecting, based at least in part on the cost, the first candidate action for further exploration in the tree structure;

associating the first candidate action with a second node of the tree structure;

determining a third candidate action for controlling motion of the vehicle relative to the second node;

inputting, into the machine learned model, a second representation of the environment associated with the second node;

receiving, from the machine learned model, a fourth candidate action for controlling motion of the vehicle relative to the second node; and

updating the set of connected nodes of the tree structure based at least in part on the third candidate action and the fourth candidate action.

6 . A method comprising:

inputting, into a machine learned model, a representation of an environment;

generating, using the machine learned model, a first candidate action for controlling motion of a vehicle in the environment;

determining, based at least in part on at least one heuristic, a second candidate action for controlling motion of the vehicle in the environment;

generating, based at least in part on the first candidate action and the second candidate action, a tree structure, wherein the tree structure comprises a parent node;

creating a set of connected nodes of the tree structure based at least in part on the first candidate action and the second candidate action, wherein the first and second candidate actions correspond to child nodes of the set of connected nodes and are associated with the parent node;

determining a control trajectory for the vehicle, using the tree structure, based at least in part on determining a minimum cost traversal associated with the set of connected nodes in the tree structure; and

controlling the vehicle in the environment, based at least in part on the control trajectory.

7 . The method of claim 6 , further comprising:

determining, based at least in part on the environment, a tree search exploration policy; and

determining the second candidate action based at least in part on the tree search exploration policy.

8 . The method of claim 6 , wherein generating the tree structure comprises:

associating a current state of the vehicle with a first node of the tree structure;

selecting, based at least in part on a cost, the first candidate action for further exploration in the tree structure;

associating the first candidate action with a second node of the tree structure;

inputting, into the machine learned model, a second representation of the environment associated with the second node; and

receiving, from the machine learned model, a third candidate action for controlling motion of the vehicle relative to the second node.

9 . The method of claim 6 , wherein generating the tree structure comprises:

associating a current state of the vehicle with a first node of the tree structure;

determining a confidence level associated with the first candidate action received from the machine learned model; and

determining, based at least in part on the confidence level, a number of additional candidate actions associated with the first node of the tree structure.

10 . The method of claim 6 , further comprising:

determining a tree search exploration policy based at least in part on a node level of a node in the tree structure, wherein the tree search exploration policy comprises a number of candidate actions; and

determining the second candidate action based at least in part on the tree search exploration policy.

11 . The method of claim 6 , wherein the machine learned model is at least one of:

an imitation vehicle trajectory model, trained based on at least in part on vehicle log data;

a reinforced learning vehicle trajectory model, trained using a first value maximization function; or

an inverse reinforced learning vehicle trajectory model, trained using a second value maximization function.

12 . The method of claim 6 , wherein:

the machine learned model is trained to output a predicted trajectory of the vehicle, based at least in part on the representation of the environment; and

the first candidate action is based at least in part on the predicted trajectory.

13 . The method of claim 12 , wherein:

the representation of the environment into to the machine learned model is based at least in part on:

a static representation of the environment at a first time associated with a first node of the tree structure;

a vehicle state of the vehicle at the first time associated with the first node; and

an agent state of an agent in the environment at the first time associated with the first node, and

an output of the machine learned model comprises:

a predicted vehicle state of the vehicle at a second time associated with a second node of the tree structure; and

a predicted agent state of an agent in the environment at the second time associated with the second node of the tree structure.

14 . The method of claim 6 , wherein:

the first candidate action is determined subsequent to receiving the representation of the environment; and

the second candidate action is determined, based at least in part on a search exploration policy, prior to receiving the representation of the environment.

15 . The method of claim 6 , wherein:

a current state of the vehicle is associated with a root node of the tree structure;

the first candidate action is associated with a first action node of the tree structure; and

the second candidate action is associated with a second action node of the tree structure.

16 . One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:

inputting, into a machine learned model, a representation of an environment;

generating, using the machine learned model, a first candidate action for controlling motion of a vehicle in the environment;

determining, based at least in part on at least one heuristic, a second candidate action for controlling motion of the vehicle in the environment;

generating, based at least in part on the first candidate action and the second candidate action, a tree structure, wherein the tree structure comprises a parent node;

creating a set of connected nodes of the tree structure based at least in part on the first candidate action and the second candidate action, wherein the first and second candidate actions correspond to child nodes of the set of connected nodes and are associated with the parent node;

determining a control trajectory for the vehicle, using the tree structure, based at least in part on determining a minimum cost traversal associated with the set of connected nodes in the tree structure; and

controlling the vehicle in the environment, based at least in part on the control trajectory.

17 . The one or more non-transitory computer-readable media of claim 16 , the operations further comprising:

determining, based at least in part on the environment, a tree search exploration policy; and

determining the second candidate action based at least in part on the tree search exploration policy.

18 . The one or more non-transitory computer-readable media of claim 16 , wherein generating the tree structure comprises:

associating a current state of the vehicle with a first node of the tree structure;

selecting, based at least in part on a cost, the first candidate action for further exploration in the tree structure;

associating the first candidate action with a second node of the tree structure;

inputting, into the machine learned model, a second representation of the environment associated with the second node; and

receiving, from the machine learned model, a third candidate action for controlling motion of the vehicle relative to the second node.

19 . The one or more non-transitory computer-readable media of claim 16 , wherein generating the tree structure comprises:

associating a current state of the vehicle with a first node of the tree structure;

determining a confidence level associated with the first candidate action received from the machine learned model; and

determining, based at least in part on the confidence level, a number of additional candidate actions associated with the first node of the tree structure.

20 . The one or more non-transitory computer-readable media of claim 16 , the operations further comprising:

determining a tree search exploration policy based at least in part on a node level of a node in the tree structure, wherein the tree search exploration policy comprises a number of candidate actions; and

determining the second candidate action based at least in part on the tree search exploration policy.

Assignments (1)

References Cited (10)