Granted Patent B2

US 12,345,821 · App. 17/484,169 · Granted Jul 1, 2025

Inventors: Arthur Daniel Costea (San Mateo, CA); David Pfeiffer (Foster City, CA); Zeng Wang (Foster City, CA); Allan Zelener (San Mateo, CA)

Assignee: Zoox, Inc.

G01S17/931G01B21/16G01S7/4863G01S17/06G06N20/00G01B11/026G06F16/786

Abstract

Claims (65)

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving first lidar data representing a physical environment at a first time, the first lidar data associated with a first position;

receiving second lidar data representing the physical environment at a second time after the first time, the second lidar data associated with a second position;

determining, based on the first lidar data, first top-down data including a first minimum height value associated with a portion of the first lidar data and a first maximum height value associated with the portion of the first portion of the first lidar data;

determining based on the second lidar data, second top-down data including representing a second minimum height value associated with a portion of the second lidar data and representing a second maximum height value associated with the portion of the second lidar data;

aligning the first top-down data with the second top-down data such that the portion of the first lidar data aligns with the portion of the second lidar data;

determining, based on the alignment, aggregated top-down data wherein the aggregated top-down data includes a multichannel top down representation of the physical environment, a first channel of the multichannel top down representation including absolute minimum height values and a second channel of the multichannel top down representation including maximum height values;

determining, based at least in part on the aggregated top-down data, object data associated with an object in the physical environment; and

performing, based at least in part on the object data, an operation associated with an autonomous vehicle.

2. The system as recited in claim 1 , wherein determining the object data further comprises:

inputting the aggregated top-down data into a machine learned model; and

receiving, from the machine learned model, the object data.

3. The system as recited in claim 1 , wherein a third channel of the multichannel top down representation includes average intensity values for the height dimension.

4. The system as recited in claim 1 , wherein the operations further comprising:

receiving third lidar data representing the physical environment at a third time after the second time, the third lidar data associated with a third position;

determining based on the third lidar data, third top-down data including representing a third minimum height value associated with a portion of the third lidar data and representing a third maximum height value associated with the portion of the third lidar data;

aligning the third top-down data with the first top-down data and the second top-down data such that the portion of the third lidar data aligns with the portion of the first lidar data and the portion of the second lidar data; and

wherein the aggregated top-down data includes the portion of the third lidar data.

5. The system as recited in claim 1 , wherein the transformation is based at least in part on a first pose of the autonomous vehicle at the first location and a second pose of the autonomous vehicle at the second location.

6. A method comprising:

receiving first lidar data representing a physical environment at a first time;

receiving second lidar data representing the physical environment at a second time;

determining, based at least in part on a first location associated with the first lidar data and a second location associated with the second lidar data, a transfer function;

determining, based at least in part on the first lidar data, the second lidar data, and the transfer function, aggregated data representing the physical environment, wherein a first channel of the aggregated data represents a first height value for a portion represented in the first lidar data and the second lidar data and a second channel of the aggregated data represents a second height value for the portion; and

determining, based at least in part on the aggregated data, object data associated with an object in the physical environment.

7. The method as recited in claim 6 , wherein determining the object data associated with the object in the physical environment further comprises:

inputting the aggregated data into a machine learned model; and

receiving, from the machine learned model, the object data.

8. The method as recited in claim 6 , further comprising:

determining an empty space associated with the physical environment based at least in part on the second lidar data; and

filtering the first lidar data to remove data within the empty space prior to generating the transform function.

9. The method as recited in claim 8 , further comprising:

generating a first mesh associated with the first lidar data;

generating a second mesh associated with the second lidar data; and

wherein determining the aggregated data comprises applying the transform function to the first mesh.

10. The method as recited in claim 6 , wherein the aggregated data is top-down aggregated data.

11. The method as recited in claim 6 , further comprising:

receiving third lidar data representing the physical environment at a third time; and

wherein the aggregated data representing is based at least in part on the third lidar data.

12. The method as recited in claim 11 , further comprising:

determining, based at least in part on a third location associated with the third lidar data and the second location associated with the second lidar data, a second transfer function; and

wherein the aggregated data representing is based at least in part on the second transfer function.

13. The method as recited in claim 6 , wherein the first height value is a maximum height value and the second height value is a minimum height value.

14. One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations comprising:

receiving first lidar data representing a physical environment at a first time;

receiving second lidar data representing the physical environment at a second time, the second time subsequent to the first time;

determining, based at least in part on a first location associated with the first lidar data and a second location associated with the second lidar data, a transfer function;

determining, based at least in part on the first lidar data, the second lidar data, and the transfer function, aggregated data representing the physical environment, wherein a first channel of the aggregated data represents a first height value for a portion represented in the first lidar data and the second lidar data and a second channel of the aggregated data represents a second height value for the portion; and

determining, based at least in part on the aggregated top-down data, object data associated with an object in the physical environment.

15. The one or more non-transitory computer-readable media as recited in claim 14 , wherein the aggregated data is a multichannel top down representation of the physical environment.

16. The one or more non-transitory computer-readable media as recited in claim 14 , wherein the operations further comprise:

determining an empty space associated with the physical environment based at least in part on the second lidar data; and

filtering the first lidar data to remove data within the empty space prior to generating the transform function.

17. The one or more non-transitory computer-readable media as recited in claim 16 , wherein the operations further comprises:

generating a first mesh associated with the first lidar data;

generating a second mesh associated with the second lidar data; and

wherein determining the aggregated data comprises applying the transform function to the first mesh.

18. The one or more non-transitory computer-readable media as recited in claim 14 , wherein the first height value is a maximum height value and the second height value is a minimum height value.

19. The one or more non-transitory computer-readable media as recited in claim 14 , the operations further comprising:

receiving third lidar data representing the physical environment at a third time, the third preceding the second time;

determining, based at least in part on a third location associated with the third lidar data and the second location associated with the second lidar data, a second transfer function; and

wherein the aggregated data representing is based at least in part on the third lidar data and the second transfer function.

20. The one or more non-transitory computer-readable media as recited in claim 14 , the operations further comprising performing, based at least in part on the object data, at least one operation associated with an autonomous vehicle.

Assignments (1)

Continuity (1)

References Cited (11)