Granted Patent B1

US 12,371,059 · App. 17/816,160 · Granted Jul 29, 2025

Inventors: Eric Yan Tin Chu (San Jose, CA); Andrew Glen Tsao (San Carlos, CA)

Assignee: Zoox, Inc.

B60W60/0015G07C5/085B60W2552/35B60W2552/53B60W2554/4042B60W2554/4043B60W2554/4044B60W2555/60

Abstract

Claims (116)

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving log data associated with a vehicle, the log data representing a scenario of the vehicle operating in an environment;

determining, based at least in part on the log data, a plurality of frames of the scenario, wherein individual frames of the plurality of frames comprise respective top-down representations of the scenario at respective times;

determining, for individual frames of the plurality of frames, a respective set of discrete areas of the environment based on a respective position of the vehicle in the individual frame, the respective set of discrete areas of the environment being hexagonal areas;

determining, for individual frames of the plurality of frames, a respective set of labels for the set of discrete areas for the individual frame based at least in part on the respective position and heading of the vehicle in the individual frame;

determining, for individual frames of the plurality of frames, respective agent feature values of agent features for individual discrete areas in the individual frame;

determining a feature vector representation of the scenario by aggregating agent feature values of the individual frames based on the labels for the discrete areas for the frame; and

determining a second scenario by comparing the feature vector representation of the scenario to a second feature vector representation of the second scenario using a threshold distance.

2. The system as recited in claim 1 , wherein:

the determining, for individual frames of the plurality of frames, of the respective set of labels for the discrete areas for the individual frame determines labels for a center discrete area and one or more concentric rings of discrete areas around the center discrete area;

a first discrete area is a first center discrete area in the determining of the labels in a first individual frame of the plurality of frames based on a first position of the vehicle in the first individual frame; and

a second discrete area is a second center discrete area in the determining of the labels in a second individual frame of the plurality of frames based on a second position of the vehicle in the second individual frame.

3. The system as recited in claim 1 , wherein the agent features include at least one of:

an agent presence feature;

an agent class feature;

an agent speed feature;

an agent heading feature;

a longitudinal agent speed feature;

a latitudinal agent speed feature;

an agent acceleration feature; or

an agent yaw rate feature.

4. The system as recited in claim 1 , the operations further comprising:

determining, for individual frames of the plurality of frames, respective road feature values of road features for individual discrete areas; and

determining the feature vector representation of the scenario at least in part by aggregating road feature values of the individual frames based on the labels for the discrete areas for the frames.

5. The system as recited in claim 4 , wherein the road features include at least one of:

a crosswalk feature;

a drivable surface feature;

a before intersection feature;

an after intersection feature;

an intersection feature;

a parking space feature;

a speed bump feature;

a driveway feature;

a road signage feature; or

a road marking feature.

6. A method comprising:

receiving log data associated with a vehicle, the log data representing a scenario of the vehicle operating in an environment;

determining, based at least in part on the log data, a frame of the scenario;

determining, for the frame of the scenario, a set of discrete areas of the environment based on a position of the vehicle in the frame;

determining, for the frame of the scenario, a set of labels for the discrete areas of the frame based at least in part on the position and a heading of the vehicle in the frame;

determining, for the frame of the scenario, respective agent feature values of agent features for agents in individual discrete areas of the discrete areas of the frame;

determining a feature vector representation of the scenario based at least in part on agent feature values of the frame and based at least in part on the labels for the discrete areas of the frame; and

determining a second scenario by comparing the feature vector representation of the scenario to a second feature vector representation of the second scenario using a threshold distance.

7. The method of claim 6 , wherein the frame comprises a top-down representation of the scenario at a corresponding time.

8. The method of claim 6 ,

further comprising performing simulation testing of the vehicle in the second scenario.

9. The method of claim 6 , further comprising:

determining, based at least in part on the log data, a plurality of additional frames of the scenario, wherein a plurality of frames of the scenario include the frame and the plurality of additional frames;

determining, for individual additional frames of the plurality of frames, a respective set of additional discrete areas of the environment based on a respective position of the vehicle in the individual additional frame;

determining, for individual additional frames of the plurality of frames, a respective set of labels for the discrete areas of the individual additional frame based at least in part on the respective position and a respective heading of the vehicle in the individual additional frame; and

determining, for individual additional frames of the plurality of frames, respective agent feature values of agent features for agents in individual discrete areas of the discrete areas of the individual additional frame; and

wherein determining the feature vector representation of the scenario further comprises aggregating agent feature values of the individual additional frames to the feature vector based on the labels for the discrete areas of the individual additional frames.

10. The method of claim 9 , wherein:

the set of discrete areas of the environment are hexagonal areas;

the determining, for individual frames of the plurality of frames, of the respective set of labels for the discrete areas for the individual frame determines labels for a center discrete area and one or more concentric rings of discrete areas around the center discrete area;

a first discrete area is a first center discrete area in the determining of the labels in a first individual frame of the plurality of frames based on a first position of the vehicle in the first individual frame; and

a second discrete area is a second center discrete area in the determining of the labels in a second individual frame of the plurality of frames based on a second position of the vehicle in the second individual frame.

11. The method of claim 9 , further comprising:

determining, for individual frames of the plurality of frames, respective road feature values of road features for individual discrete areas; and

determining the feature vector representation of the scenario at least in part by aggregating road feature values of the individual frames based on the labels for the discrete areas for the frames.

12. The method of claim 11 , wherein the road features include at least one of:

a crosswalk feature;

a drivable surface feature;

a before intersection feature;

an after intersection feature;

an intersection feature;

a parking space feature;

a speed bump feature;

a driveway feature;

a road signage feature; or

a road marking feature.

13. The method of claim 6 , wherein the agent features include at least one of:

an agent presence feature;

an agent class feature;

an agent speed feature;

an agent heading feature;

a longitudinal agent speed feature;

a latitudinal agent speed feature;

an agent acceleration feature; or

an agent yaw rate feature.

14. One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving log data associated with a vehicle, the log data representing a scenario of the vehicle operating in an environment;

determining, based at least in part on the log data, a frame of the scenario;

determining, for the frame of the scenario, a set of discrete areas of the environment based on a position of the vehicle in the frame;

determining, for the frame of the scenario, a set of labels for the discrete areas for the frame based at least in part on the position and a heading of the vehicle in the frame;

determining, for the frame of the scenario, respective agent feature values of agent features for agents in individual discrete areas of the discrete areas of the frame;

determining a feature vector representation of the scenario based at least in part on agent feature values of the frame and based at least in part on the labels for the discrete areas for the frame; and

determining a second scenario by comparing the feature vector representation of the scenario to a second feature vector representation of the second scenario using a threshold distance.

15. The one or more non-transitory computer-readable media of claim 14 , wherein the frame comprises a top-down representation of the scenario at a corresponding time.

16. The one or more non-transitory computer-readable media of claim 14 ,

further comprising performing simulation testing of the vehicle in the second scenario.

17. The one or more non-transitory computer-readable media of claim 14 , further comprising:

determining, based at least in part on the log data, a plurality of additional frames of the scenario, wherein a plurality of frames of the scenario include the frame and the plurality of additional frames;

determining, for individual additional frames of the plurality of frames, a respective set of additional discrete areas of the environment based on a respective position of the vehicle in the individual additional frame;

determining, for individual additional frames of the plurality of frames, a respective set of labels for the discrete areas of the individual additional frame based at least in part on the respective position and a respective heading of the vehicle in the individual additional frame; and

determining, for individual additional frames of the plurality of frames, respective agent feature values of agent features for agents in individual discrete areas of the discrete areas of the individual additional frame; and

wherein determining the feature vector representation of the scenario further comprises aggregating agent feature values of the individual additional frames to the feature vector based on the labels for the discrete areas of the individual additional frames.

18. The one or more non-transitory computer-readable media of claim 17 , wherein:

the set of discrete areas of the environment are hexagonal areas;

the determining, for individual frames of the plurality of frames, of the respective set of labels for the discrete areas for the individual frame determines labels for a center discrete area and one or more concentric rings of discrete areas around the center discrete area;

a first discrete area is a first center discrete area in the determining of the labels in a first individual frame of the plurality of frames based on a first position of the vehicle in the first individual frame; and

a second discrete area is a second center discrete area in the determining of the labels in a second individual frame of the plurality of frames based on a second position of the vehicle in the second individual frame.

19. The one or more non-transitory computer-readable media of claim 17 , the operations further comprising:

determining, for individual frames of the plurality of frames, respective road feature values of road features for individual discrete areas; and

determining the feature vector representation of the scenario at least in part by aggregating road feature values of the individual frames based on the labels for the discrete areas for the frames.

20. The one or more non-transitory computer-readable media of claim 14 , wherein the agent features include at least one of:

an agent presence feature;

an agent class feature;

an agent speed feature;

an agent heading feature;

a longitudinal agent speed feature;

a latitudinal agent speed feature;

an agent acceleration feature; or

an agent yaw rate feature.

Assignments (1)

References Cited (20)