Granted Patent B1

US 12,491,871 · App. 18/375,292 · Granted Dec 9, 2025

Inventors: Linjun Zhang (Foster City, CA); Marin Kobilarov (Baltimore, MD)

Assignee: Zoox, Inc.

B60W30/09B60W30/0956B60W40/105B60W40/107B60W50/0097B60W60/0015B60W60/00274B60W2050/0031B60W2554/4045B60W2554/80

Abstract

Claims (77)

1 . A system comprising:

one or more processors; and

one or more non-transitory memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:

detecting an object proximate a vehicle traversing an environment;

determining a set of vehicle states associated with a path for controlling the vehicle;

determining a set of object states associated with a predicted trajectory of the object;

determining a set of interaction scores based at least in part on the set of vehicle states and the set of object states;

determining, based at least in part on the set of interaction scores and a first parallel reduction by separate threads of a processing unit of the set of object states, a trajectory importance score associated with the predicted trajectory;

determining, based at least in part on the trajectory importance score and a second parallel reduction by the separate threads of the processing unit of the trajectory importance score, an object importance score associated with the object; and

controlling the vehicle based at least in part on the object importance score.

2 . The system of claim 1 , wherein the predicted trajectory is a preliminary predicted trajectory and controlling the vehicle based at least in part on the object importance score comprises one of:

determining, based at least in part on the object importance score being greater than or equal to a threshold object importance score, a second predicted trajectory using a machine learned model based at least in part on object detection data associated with the object and a candidate action for controlling the vehicle,

wherein controlling the vehicle is based at least in part on the second predicted trajectory; or

determining that the object importance score is less than or equal to a threshold, wherein:

controlling the vehicle is based at least in part on the preliminary predicted trajectory, and

the preliminary predicted trajectory is determined by a kinematic model.

3 . The system of claim 1 , wherein:

a first processing unit thread determines a first interaction score of the set of interaction scores and stores the first interaction score in a first range of memory associated with interactions scores determined for the object and the predicted trajectory; and

a second processing unit thread determines a second interaction score of the set of interaction scores and stores the second interaction score in the first range of memory.

4 . The system of claim 3 , wherein a row of memory elements is associated with a single trajectory for the object and columns associated with the row are associated with different vehicle states.

5 . The system of claim 1 , wherein:

determining a first vehicle state of the set of vehicle states comprises determining, based at least in part on at least one of a speed or acceleration of the vehicle, a minimum time and a maximum time that the vehicle will reach a first location along the path; and

determining a first object state of the set of object states comprises determining, based at least in part on a buffer time, a soonest time of arrival at a second location along the predicted trajectory and a latest time of arrival at the second location.

6 . The system of claim 1 , wherein:

determining the object importance score is based at least in part on a second trajectory importance score associated with a second predicted trajectory determined for the object.

7 . One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

detecting an object proximate a vehicle traversing an environment;

determining a set of vehicle states associated with a path for controlling the vehicle;

determining a set of object states associated with a predicted trajectory of the object;

determining a set of interaction scores based at least in part on the set of vehicle states and the set of object states;

determining, based at least in part on the set of interaction scores and a first parallel reduction by separate threads of a processing unit of the set of object states, a trajectory importance score associated with the predicted trajectory;

determining, based at least in part on the trajectory importance score and a second parallel reduction by the separate threads of the processing unit of the trajectory importance score, an object importance score associated with the object; and

controlling the vehicle based at least in part on the object importance score.

8 . The one or more non-transitory computer-readable media of claim 7 , wherein the predicted trajectory is a preliminary predicted trajectory and controlling the vehicle based at least in part on the object importance score comprises one of:

determining, based at least in part on the object importance score being greater than or equal to a threshold object importance score, a second predicted trajectory using a machine learned model based at least in part on object detection data associated with the object and a candidate action for controlling the vehicle,

wherein controlling the vehicle is based at least in part on the second predicted trajectory; or

determining that the object importance score is less than or equal to a threshold, wherein:

controlling the vehicle is based at least in part on the preliminary predicted trajectory, and

the preliminary predicted trajectory is determined by a kinematic model.

9 . The one or more non-transitory computer-readable media of claim 8 , wherein controlling the vehicle comprises determining the second predicted trajectory and determining the second predicted trajectory comprises:

determining a set of locations in the environment associated with interaction scores that meet or exceed a threshold interaction score; and

determining, by a machine-learned model and based at least in part on the candidate action for controlling the vehicle, a set of likelihoods associated with the set of locations, wherein a first likelihood of the set of likelihoods indicates a likelihood the object will occupy a first location of the set of locations.

10 . The one or more non-transitory computer-readable media of claim 7 , wherein:

a first processing unit thread determines a first interaction score of the set of interaction scores and stores the first interaction score in a first range of memory associated with interactions scores determined for the object and the predicted trajectory; and

a second processing unit thread determines a second interaction score of the set of interaction scores and stores the second interaction score in the first range of memory.

11 . The one or more non-transitory computer-readable media of claim 10 , wherein a row of memory elements is associated with a single trajectory for the object and columns associated with the row are associated with different vehicle states.

12 . The one or more non-transitory computer-readable media of claim 7 , wherein:

determining a first vehicle state of the set of vehicle states comprises determining, based at least in part on at least one of a speed or acceleration of the vehicle, a minimum time and a maximum time that the vehicle will reach a first location along the path; and

determining a first object state of the set of object states comprises determining, based at least in part on a buffer time, a soonest time of arrival at a second location along the predicted trajectory and a latest time of arrival at the second location.

13 . The one or more non-transitory computer-readable media of claim 7 , wherein:

determining the object importance score is based at least in part on a second trajectory importance score associated with a second predicted trajectory determined for the object.

14 . A method comprising:

detecting an object proximate a vehicle traversing an environment;

determining a set of vehicle states associated with a path for controlling the vehicle;

determining a set of object states associated with a predicted trajectory of the object;

determining a set of interaction scores based at least in part on the set of vehicle states and the set of object states;

determining, based at least in part on the set of interaction scores and a first parallel reduction by separate threads of a processing unit of the set of object states, a trajectory importance score associated with the predicted trajectory;

determining, based at least in part on the trajectory importance score and a second parallel reduction by the separate threads of the processing unit of the trajectory importance score, an object importance score associated with the object; and

controlling the vehicle based at least in part on the object importance score.

15 . The method of claim 14 , wherein the predicted trajectory is a preliminary predicted trajectory and controlling the vehicle based at least in part on the object importance score comprises one of:

determining, based at least in part on the object importance score being greater than or equal to a threshold object importance score, a second predicted trajectory using a machine learned model based at least in part on object detection data associated with the object and a candidate action for controlling the vehicle,

wherein controlling the vehicle is based at least in part on the second predicted trajectory; or

determining that the object importance score is less than or equal to a threshold, wherein:

controlling the vehicle is based at least in part on the preliminary predicted trajectory, and

the preliminary predicted trajectory is determined by a kinematic model.

16 . The method of claim 15 , wherein controlling the vehicle comprises determining the second predicted trajectory and determining the second predicted trajectory comprises:

determining a set of locations in the environment associated with interaction scores that meet or exceed a threshold interaction score; and

determining, by a machine-learned model and based at least in part on the candidate action for controlling the vehicle, a set of likelihoods associated with the set of locations, wherein a first likelihood of the set of likelihoods indicates a likelihood the object will occupy a first location of the set of locations.

17 . The method of claim 14 , wherein:

a first processing unit thread determines a first interaction score of the set of interaction scores and stores the first interaction score in a first range of memory associated with interactions scores determined for the object and the predicted trajectory; and

a second processing unit thread determines a second interaction score of the set of interaction scores and stores the second interaction score in the first range of memory.

18 . The method of claim 17 , wherein a row of memory elements is associated with a single trajectory for the object and columns associated with the row are associated with different vehicle states.

19 . The method of claim 14 , wherein:

determining a first vehicle state of the set of vehicle states comprises determining, based at least in part on at least one of a speed or acceleration of the vehicle, a minimum time and a maximum time that the vehicle will reach a first location along the path; and

determining a first object state of the set of object states comprises determining, based at least in part on a buffer time, a soonest time of arrival at a second location along the predicted trajectory and a latest time of arrival at the second location.

20 . The method of claim 14 , wherein:

determining the object importance score is based at least in part on a second trajectory importance score associated with a second predicted trajectory determined for the object.

Assignments (1)

References Cited (11)

Cited By (1)

US 12,735,065