Granted Patent B1

US 12,553,739 · App. 18/510,480 · Granted Feb 17, 2026

Inventors: Arnau Franci Rodon (San Francisco, CA); Soroush Dean Khadem (San Francisco, CA); Till Kroeger (Chicago, IL); Weiying Wang (Cambridge, MA)

Assignee: Zoox, Inc.

G01C21/3841

Abstract

Claims (64)

1 . A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions that, when executed, cause the system to perform operations comprising:

receiving first sensor data associated with a first vehicle pose and second sensor data associated with a second vehicle pose;

identifying a first landmark in the first sensor data and in the second sensor data;

determining, based at least in part on the first landmark, the first vehicle pose, and the second vehicle pose, a first alignment between the first sensor data and the second sensor data;

providing, based at least in part on the first alignment between the first sensor data and the second sensor data, representations of the first sensor data and the second sensor data to a user interface;

receiving, via the user interface, user input indicating a position associated with a user-selected feature represented in the first sensor data and in the second sensor data;

determining, based at least in part on the position associated with the user-selected feature, the first vehicle pose, and the second vehicle pose, a second alignment between the first sensor data and the second sensor data;

determining map data based at least in part on the first sensor data, the second sensor data, and the second alignment between the first sensor data and the second sensor data; and

transmitting the map data to a vehicle configured to be controlled based at least in part on the map data.

2 . The system of claim 1 , the operations further comprising:

determining a confidence associated with the first vehicle pose or the second vehicle pose based at least in part on the first alignment between the first sensor data and the second sensor data; and

comparing the confidence associated with the first vehicle pose or the second vehicle pose to a confidence threshold,

wherein providing the first sensor data and the second sensor data to a user interface is based at least in part on the comparing the confidence associated with the first vehicle pose or the second vehicle pose to the confidence threshold.

3 . The system of claim 1 , the operations comprising:

determining, based at least in part on the user input, an uncertainty associated with the position associated with the user-selected feature; and

determining the map data based at least in part on the uncertainty.

4 . The system of claim 3 , the operations comprising:

displaying, on the user interface, an indication of the uncertainty;

receiving, via the user interface, a further user input indicating the position associated with the user-selected feature; and

updating the uncertainty associated with the position.

5 . The system of claim 1 , the operations comprising:

receiving a first user input on the representation of the first sensor data, the first user input indicating the position of the user-selected feature in the representation of the first sensor data;

receiving a second user input on the representation of the second sensor data, the second user input indicating the position of the user-selected feature in the representation of the first sensor data; and

determining the second alignment between the first sensor data based at least in part on the first user input and the second user input.

6 . A method comprising:

receiving first sensor data associated with a first vehicle pose and second sensor data associated with a second vehicle pose, the first vehicle pose and the second vehicle pose separated by a distance greater than a threshold distance;

receiving, via a user interface, a user input indicating a position associated with a feature represented in the first sensor data and the second sensor data;

determining, based at least in part on the position associated with the feature, the first vehicle pose, and the second vehicle pose, an alignment between the first sensor data and the second sensor data;

determining map data based at least in part on the first sensor data, the second sensor data, and the alignment; and

transmitting the map data to a vehicle configured to be controlled based at least in part on the map data.

7 . The method of claim 6 , wherein the first sensor data is associated with a first time, the second sensor data is associated with a second time, and wherein the second time is at least a threshold time period after the first time.

8 . The method of claim 6 , wherein the first sensor data is associated with a different sensor system than the second sensor data.

9 . The method of claim 6 , wherein the first sensor data is associated with a different vehicle than the second sensor data.

10 . The method of claim 6 , comprising:

receiving initial map data;

determining that a confidence metric associated with a first region represented in the map data is below a threshold confidence, the first and second sensor data associated with the first region; and

presenting the first and second sensor data on the user interface based at least in part on determining that the confidence metric is below the threshold confidence.

11 . The method of claim 6 , comprising:

receiving a first user input indicating a first position of the feature in a two-dimensional representation of the first sensor data;

determining a first ray in three-dimensional space associated with the first position;

receiving a second user input indicating a second position of the feature in a two-dimensional representation of the second sensor data;

determining a second ray in three-dimensional space associated with the second position;

determining a third position in three-dimensional space based at least in part on a first distance from the third position to the first ray and on a second distance from the third position to the second ray; and

identifying the third position as the position of the feature.

12 . The method of claim 11 , further comprising presenting a projection of the third position on the two-dimensional representation of the first sensor data or second sensor data.

13 . The method of claim 12 , further comprising:

determining an error associated with the third position; and

presenting a representation of the error on the two-dimensional representation of the first sensor data or second sensor data.

14 . The method of claim 6 , wherein determining the alignment comprises perturbing the first vehicle pose or the second vehicle pose based at least in part on the position associated with the feature.

15 . One or more non-transitory computer-readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:

receiving first sensor data associated with a first vehicle pose and second sensor data associated with a second vehicle pose, the first vehicle pose and the second vehicle pose separated by a distance greater than a threshold distance;

receiving, via a user interface, a user input indicating a position associated with a feature represented in the first sensor data and the second sensor data;

determining, based at least in part on the position associated with the feature, the first vehicle pose, and the second vehicle pose, an alignment between the first sensor data and the second sensor data;

determining map data based at least in part on the first sensor data, the second sensor data, and the alignment; and

transmitting the map data to a vehicle configured to be controlled based at least in part on the map data.

16 . The one or more non-transitory computer-readable media of claim 15 , the operations comprising:

presenting, on the user interface, a representation of the first sensor data and an indication of a suggested position or area represented in the first sensor data and the second sensor data.

17 . The one or more non-transitory computer-readable media of claim 15 , wherein the first sensor data or second sensor data comprise at least one of: camera data, radar data, or lidar data.

18 . The one or more non-transitory computer-readable media of claim 15 , the operations further comprising:

determining a confidence metric associated with the map data;

determining that the confidence metric does not satisfy a confidence condition; and

receiving the user input based at least in part on determining that the confidence metric does not satisfy the confidence condition.

Assignments (1)

References Cited (10)