IP Library › Granted Patent US 12682135

Granted Patent B1

US 12682135 · App. 18/072,271 · Granted Jul 14, 2026

Controlling agent movement in driving simulations based on vehicle log data

Inventors: Tod Cameron Semple (Woodside, CA); Maxwell Chandler Robinson (Burlingame, CA)

Assignee: Zoox, Inc.

G06F30/20 G06T7/246 G06T2207/10016 G06T2207/10028 G06T2207/30252 G07C5/04

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12682135
App. No.: 18/072,271
Granted: Jul 14, 2026
Kind: B1

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Techniques are described herein for analyzing vehicle log data and determining initial movement times for stationary objects represented in the log data. Various techniques may include receiving log data including a sequence of bounding shapes representing an object detected in a physical environment, and analyzing the sequence of bounding shapes to distinguish actual object movements from noise within the perceived object data and/or errors caused by partially occluded objects. Additional techniques described herein may include executing log-based driving simulations during which simulated agents are controlled based on the initial movement times of corresponding objects in the physical environment. For instance, a simulated agent may be controlled using a trajectory of a corresponding real-world object and/or may be converted to a programmatic agent at a time during the simulation after the initial movement time of the corresponding real-world object.

Claims (92)

1 . A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the system to perform operations comprising:

receiving log data associated with an operation of a vehicle over a time period in a physical environment, the log data including:

a first bounding shape representing an object in the physical environment at a first time; and

a second bounding shape representing the object in the physical environment at a second time after the first time;

determining, based at least in part on the first bounding shape and the second bounding shape, that a perceived extent of the object in the log data at the second time is stable;

determining, on the first bounding shape, a first point and a second point;

determining, on the second bounding shape, a third point corresponding to the first point on the first bounding shape, and a fourth point corresponding to the second point on the first bounding shape;

determining a first movement amount associated with the second bounding shape, based at least in part on the first point and the third point;

determining a second movement amount associated with the second bounding shape, based at least in part on the second point and the fourth point;

determining, based at least in part on the first movement amount and the second movement amount associated with the second bounding shape, a movement start time associated with the object in the physical environment;

executing a simulation based at least in part on the log data, including controlling a simulated agent in a simulated environment, wherein the simulated environment is based at least in part on the physical environment; and

controlling the simulated agent in the simulated environment based at least in part on the movement start time associated with the object in the physical environment.

2 . The system claim 1 , wherein controlling the simulated agent comprises:

determining a first simulation time in the simulation corresponding to the movement start time associated with the object in the physical environment;

controlling the simulated agent, at a second simulation time after the first simulation time, to correspond to the object in the physical environment; and

determining, at a third simulation time after the second simulation time, a programmatic planner component for controlling the simulated agent in the simulated environment.

3 . The system of claim 1 , wherein:

determining the first point comprises determining, based at least in part on a yaw of the first bounding shape, a front point on a front surface of the first bounding shape; and

determining the second point comprises determining, based at least in part on the yaw of the first bounding shape, a back point on a back surface of the first bounding shape.

4 . The system of claim 1 , wherein:

determining the first point comprises determining a first object feature on a surface of the first bounding shape; and

determining the third point comprises determining a second object feature, corresponding to the first object feature, on the second bounding shape.

5 . The system of claim 1 , wherein determining the movement start time associated with the object comprises:

determining a first velocity associated with the first point, based at least in part on the first movement amount;

determining a second velocity associated with the second point, based at least in part on the second movement amount; and

determining that a difference between the first velocity and the second velocity is less than a velocity difference threshold.

6 . A method comprising:

receiving log data associated with an operation of a vehicle over a time period in an environment, the log data including:

first object data representing an object in the environment at a first time, the first object data having a first extent; and

second object data representing the object in the environment at a second time after the first time, the second object data having a second extent;

determining, based at least in part on the first extent and the second extent, that a representation of the object in the log data at the second time is stable;

determining, on the first object data, a first point and a second point;

determining, on the second object data, a third point and a fourth point, wherein the third point is associated with the first point on the first object data, and wherein the fourth point is associated with the second point on the first object data;

determining, based at least in part on the first point and the third point, a first movement associated with the representation of the object in the log data;

determining, based at least in part on the second point and the fourth point, a second movement associated with the representation of the object in the log data;

determining, based at least in part on the first movement and the second movement, a movement start time associated with the object in the environment;

controlling the simulated agent in the simulated environment based at least in part on the movement start time associated with the object in the physical environment.

7 . The method of claim 6 , wherein controlling the simulated agent comprises:

determining a first simulation time in the simulation corresponding to the movement start time associated with the object in the environment;

controlling the simulated agent, at a second simulation time after the first simulation time, to correspond to the object in the environment; and

determining, at a third simulation time after the second simulation time, a programmatic planner component for controlling the simulated agent in the simulated environment.

8 . The method of claim 6 , wherein the first object data comprises at least one of a first bounding shape or a first point cloud representing the object at the first time, and wherein the second object data comprises at least one of a second bounding shape or a second point cloud representing the object at the second time.

9 . The method of claim 6 , wherein:

determining the first point comprises determining a front point on a boundary of the first object data, based at least in part on a yaw associated with the first object data, and

determining the second point comprises determining a back point on the boundary of the first object data, based at least in part on the yaw associated with the first object data.

10 . The method of claim 6 , wherein:

determining the first point comprises determining a first feature on a boundary of the first object data, and

determining the third point comprises determining, on a boundary of the second object data, a second feature corresponding to the first feature.

11 . The method of claim 6 , wherein determining the movement start time associated with the object comprises:

determining a first velocity associated with the first point, based at least in part on the first movement;

determining a second velocity associated with the second point, based at least in part on the second movement; and

determining that a difference between the first velocity and the second velocity is less than a velocity difference threshold.

12 . The method of claim 6 , wherein the log data further includes third object data representing the object in the environment at a third time after the second time, and wherein determining the movement start time associated with the object comprises determining:

a first perceived distance traveled by the representation of the object during a time period between the first time and the third time; and

a second perceived distance between the representation of the object in the first object data and the representation of the object in the third object data.

13 . The method of claim 6 , wherein determining the movement start time associated with the object comprises:

determining, based at least in part on the first movement and the second movement, a perceived acceleration associated with the representation of the object;

determining a range of acceleration values associated with the object; and

comparing the perceived acceleration associated with the representation of the object to the range of acceleration values.

14 . One or more non-transitory computer-readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising: