IP Library › Granted Patent US 11,138,350
Granted Patent B2
US 11,138,350 · App. 16/163,466 · Granted Oct 5, 2021

Procedural world generation using tertiary data

Inventors: James Graham Dolan (Foster City, CA); Douglas Raymond Brooks (San Mateo, CA)
Assignee: Zoox, Inc.
G06F30/20G05D1/0088G06K9/6212G06K9/6215G06K9/6262G06K9/6269G06N3/04G06N3/08G06T15/04G06T17/05G06T17/20G06T19/20G01S13/89G01S15/89G01S17/89G06T2215/16G06T2219/2004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,138,350
App. No.
16/163,466
Granted
Oct 5, 2021
Kind
B2
Abstract

Procedural world generation using tertiary data is described. In an example, a computing device can receive road network data associated with the real environment and a road mesh associated with a real environment. The computing device can associate the road network data with the road mesh to generate a simulated environment. Additionally, the computing device can associate supplemental data with the road network data and the road mesh to enhance the simulated environment (e.g., supplementing information otherwise unavailable to the sensor data due to an occlusion). The computing device can output the simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous robotic computing device for at least one of navigating, planning, or decision making.

Claims (47)

1. A computer-implemented method comprising:

receiving sensor data from a plurality of data collection devices in a real environment;

accessing road network data associated with the real environment, the road network data based at least in part on the real environment;

determining, based at least in part on the sensor data, a road mesh associated with the real environment;

associating the road network data with the road mesh to generate a simulated environment, wherein the simulated environment is incomplete with respect to the real environment;

accessing supplemental data associated with the real environment, wherein the supplemental data provides information associated with the real environment that is unavailable to the plurality of data collection devices;

determining an error between a first portion of the supplemental data and a second portion of the road mesh, the first portion and the second portion being associated with a same region of the real environment;

determining, as a determination, that the error meets or exceeds a threshold amount of error;

adjusting at least one of the supplemental data or the road mesh to reduce the error based at least in part on the determination;

associating the supplemental data with the simulated environment to supplement the simulated environment as a modified simulated environment; and

outputting the modified simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous robotic computing device for at least one of navigating, planning, or decision making.

2. The computer-implemented method as claim 1 recites, wherein the supplemental data comprises a geospatial file format storing a raster-based digital elevation model.

3. The computer-implemented method as claim 2 recites, wherein the geospatial file format is associated with a United States Geological Survey (USGS) Data Evaluation Model (DEM) standard.

4. The computer-implemented method as claim 1 recites, wherein the simulated environment is incomplete due to an occlusion in the sensor data associated with a parked car or an alleyway.

5. The computer-implemented method as claim 1 recites, wherein the error comprises at least one of an average error associated with the same region of the real environment.

6. The computer-implemented method as claim 1 recites, further comprising applying a deformation lattice to one or more of at least a portion of the supplemental data or a corresponding portion of the road mesh to substantially align the supplemental data and the road mesh to reduce the error.

7. A system comprising:

at least one processor; and

one or more computer-readable instructions that, when executed by the at least one processor, cause the at least one processor to perform acts comprising:

receiving sensor data from at least one data collection device in a real environment;

accessing at least one of road network data associated with the real environment or a road mesh associated with the real environment, the road mesh being associated with the sensor data;

generating a simulated environment based on the at least one of the road network data or the road mesh;

associating supplemental data with the simulated environment to generate a modified simulated environment;

outputting the modified simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous robotic computing device for at least one of controlling the autonomous robotic computing device;

measuring an error between a first portion of the supplemental data and a second portion of the road mesh;

determining, as a determination, that the error meets or exceeds a threshold amount of error; and

applying a deformation lattice to one or more of the first portion or the second portion to reduce the error based at least in part on the determination.

8. The system as claim 7 recites, wherein the simulated environment is incomplete due to at least one occlusion in the real environment.

9. The system as claim 7 recites, wherein the supplemental data provides information associated with the real environment that is otherwise unavailable to the at least one data collection device due to at least one occlusion.

10. The system as claim 7 recites, wherein the road network data comprises a two-dimensional representation of the real environment and comprises at least one indication of a driving lane element, a bike lane element, a parking lane element, or a crosswalk element.

11. The system as claim 7 recites, wherein the road mesh comprises a plurality of three-dimensional tiles output from a mapping system.

12. The system as claim 7 recites, wherein the acts further comprise accessing the road network data and the road mesh and associating the road network data and the road mesh based at least in part on projecting the road network data into the road mesh.

13. The system as claim 7 recites, wherein the supplemental data comprises elevation data collected by a third-party source or system.

14. A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations comprising:

accessing at least one of road network data associated with a real environment or a road mesh associated with the real environment;

generating a simulated environment based on the at least one of the road network data or the road mesh;

associating supplemental data with the simulated environment to generate a modified simulated environment;

outputting the modified simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous robotic computing device for at least one of controlling the autonomous robotic computing device;

measuring a difference between a first portion of the supplemental data and a second portion of the road mesh, the first portion and the second portion being associated with a same region of the real environment that is devoid of objects;

determining, as a determination, that the difference meets or exceeds a threshold difference; and

applying a deformation lattice to one or more of the first portion or the second portion to reduce the difference based at least in part on the determination.

15. The non-transitory computer-readable medium as claim 14 recites, wherein the supplemental data comprises elevation data collected by a third-party source or system.

16. The non-transitory computer-readable medium as claim 14 recites, wherein the supplemental data comprises a geospatial file format that is associated with a United States Geological Survey (USGS) Data Evaluation Model (DEM) standard.

17. The non-transitory computer-readable medium as claim 14 recites, wherein the supplemental data provides information associated with the real environment that is otherwise unavailable to the at least one data collection device due to at least one occlusion.

18. The computer-implemented method as claim 1 recites, wherein the error comprises at least one of: a single measurement, an average of multiple measurements, a maximum over an area, a minimum over an area, or a total error representative of a difference between the first portion of the supplemental data and the second portion of the road mesh.

19. The system as claim 7 recites, wherein the error comprises at least one of: a single measurement, an average of multiple measurements, a maximum over an area, a minimum over an area, or a total error representative of a difference between the first portion of the supplemental data and the second portion of the road mesh.

20. The non-transitory computer-readable medium as claim 14 recites, wherein the difference comprises at least one of: a single measurement, an average of multiple measurements, a maximum over an area, a minimum over an area, or a total error representative of the difference between the first portion of the supplemental data and the second portion of the road mesh.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2018
From: DOLAN, JAMES GRAHAM; BROOKS, DOUGLAS RAYMOND
To: ZOOX, INC.
Reel/Frame 047576/0491 →
Continuity (2)
Provisional Application 62716839 · Aug 9, 2018
Related Publication 20200050716A1 · Feb 13, 2020
Cited By (1)
US 12,664,462