IP Library Granted Patent US 11,663,377
Granted Patent B2
US 11,663,377 · App. 16/586,496 · Granted May 30, 2023

Sensor arrangement validation using simulated environments

Inventors: David Bearl DeVore (Daly City, CA); Alexander Charles Granieri (Palo Alto, CA); Alfred Charles Jones, II (San Jose, CA); Han Han Xue (San Francisco, CA)
Assignee: Woven Planet North America, Inc.
G06F30/20G01S17/89G06F30/15G06T17/05G06V20/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,663,377
App. No.
16/586,496
Granted
May 30, 2023
Kind
B2
Abstract

In one embodiment, a method includes generating a simulated sensor based on performance features associated with a sensor. The method includes determining a placement indicator specifying a location and orientation of the simulated sensor on a vehicle in a virtual environment associated with the vehicle. The method includes simulating, based on the performance features associated with the sensor, behavior of one or more emissions originating from the simulated sensor in the virtual environment. For each emission, the simulating includes determining an interaction of the emission with one or more simulated objects in the virtual environment. The method includes providing a representation of a capability of the sensor based on the simulated behavior of the emissions.

Claims (53)

1. A method comprising, by a computing system:

generating a simulated sensor based on performance features associated with a sensor;

determining a placement indicator specifying a location and orientation of the simulated sensor on a vehicle in a virtual environment associated with the vehicle;

simulating, based on the performance features associated with the sensor, behavior of emissions originating from the simulated sensor in the virtual environment as rays emitted from the simulated sensor into the virtual environment as representations of a field-of-view (FOV) of the simulated sensor relative to the virtual environment, wherein the simulating comprises determining a number of the emissions that interact with the virtual environment including one or more simulated objects in the virtual environment; and

providing a representation of a capability of the sensor in the location and orientation simulated on the vehicle based on the behavior of the emissions by calculating a performance metric for the simulated sensor based on the behavior of the emissions, including a beam efficiency metric that defines a ratio of the number of emission interactions to a total number of simulated emissions to quantify wasted emissions that indicate which of the emissions did not interact with the one or more simulated objects.

2. The method of claim 1 , further comprising providing the representation of the capability of the sensor by:

generating a visual representation of the emissions interactions with the one or more simulated objects in the virtual environment.

3. The method of claim 1 , further comprising

identifying a number of the emissions that interact with the simulated sensor and the vehicle; and

calculating the performance metric as a ratio of the number of the emissions that interact with the simulated sensor and the vehicle versus a number of the emissions that interact with the virtual environment.

4. The method of claim 1 , wherein providing the representation of the capability of the sensor includes:

estimating a sensor coverage for the simulated sensor at the location and orientation based on the behavior of the emissions in the virtual environment.

5. The method of claim 4 ,

wherein estimating the sensor coverage includes determining when the sensor coverage does not satisfy a threshold sensor coverage; and

providing a notification indicating that the sensor coverage does not satisfy the threshold sensor coverage.

6. The method of claim 4 , further comprising:

generating a visual representation of the sensor coverage of the simulated sensor.

7. The method of claim 4 , further comprising providing the representation of the capability of the sensor by:

identifying one or more areas of the virtual environment beyond the sensor coverage of the simulated sensor at the placement indicator based on the behavior.

8. The method of claim 1 , further comprising:

determining a sensor type of the simulated sensor, wherein:

simulating behavior of the emissions originating from the simulated sensor in the virtual environment further comprises:

projecting each emission from the simulated sensor into the virtual environment wherein properties of projecting, including at least a direction of the emission, are determined based on the sensor type; and

detecting a point of interaction between the emissions and a surface corresponding to a simulated object in the virtual environment.

9. The method of claim 8 , wherein a number of emissions projected in a particular portion of the virtual environment over a particular time window is determined based on the performance features associated with the sensor.

10. The method of claim 1 , further comprising providing the representation of the capability of the sensor by:

generating a plurality of virtual environments by procedurally generating an arrangement of a plurality of objects in each virtual environment;

simulating behavior of emissions originating from the simulated sensor in each virtual environment of the plurality of virtual environments; and

combining a representation of the capability of the sensor determined with respect to each virtual environment.

11. The method of claim 1 , further comprising generating the virtual environment by:

constructing a virtual environment encompassing a vehicle operation scenario; and

modeling activity of one or more objects in the virtual environment, wherein the activity simulates the behavior of the one or more objects in the vehicle operation scenario.

12. The method of claim 11 , wherein the representation of the capability of the sensor includes a sensor coverage area associated with the sensor recommended for an autonomous vehicle to navigate the vehicle operation scenario.

13. The method of claim 11 , wherein the representation of the capability of the sensor includes a performance of the sensor with respect to the one or more objects in the vehicle operation scenario.

14. The method of claim 1 , further comprising modeling a location and orientation of the simulated sensor in the virtual environment by:

accessing a model of the vehicle having an arrangement of discrete sensor placement locations, wherein an arrangement of discrete sensor placement locations of the model of the vehicle corresponds to the arrangement of the discrete sensor placement locations of the vehicle; and

positioning the simulated sensor on the model of the vehicle among the arrangement of discrete sensor placement locations.

15. The method of claim 14 , further comprising:

recommending a location and orientation of the sensor on the vehicle based on an optimization of the representation of the capability of the sensor.

16. The method of claim 1 , further comprising:

modeling the simulated sensor in the virtual environment based on the location and orientation of the simulated sensor specified in the placement indicator.

17. A system comprising: one or more processors and one or more computer-readable non-transitory storage media coupled to the one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to:

generate a simulated sensor based on performance features associated with a sensor;

determine a placement indicator specifying a location and orientation of the simulated sensor on a vehicle in a virtual environment associated with the vehicle;

simulate, based on the performance features associated with the sensor, behavior of emissions originating from the simulated sensor in the virtual environment as rays emitted from the simulated sensor into the virtual environment as representations of a field-of-view (FOV) of the simulated sensor relative to the virtual environment, wherein the instructions to simulate include instructions to determine a number of the emissions that interact with the virtual environment including one or more simulated objects in the virtual environment; and

provide a representation of a capability of the sensor in the location and orientation simulated on the vehicle based on the behavior of the emissions by calculating a performance metric for the simulated sensor based on the behavior of the emissions, including a beam efficiency metric that defines a ratio of the number of emission interactions to a total number of simulated emissions to quantify wasted emissions that indicate which of the emissions did not interact with the one or more simulated objects.

18. The system of claim 17 , wherein the instructions are further operable when executed by one or more of the processors to cause the system to provide the representation of the capability of the sensor by:

generating a visual representation of the emissions interactions with the one or more simulated objects in the virtual environment.

19. One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors to perform operations comprising:

generate a simulated sensor based on performance features associated with a sensor;

determine a placement indicator specifying a location and orientation of the simulated sensor on a vehicle in a virtual environment associated with the vehicle;

simulate, based on the performance features associated with the sensor, behavior of emissions originating from the simulated sensor in the virtual environment as rays emitted from the simulated sensor into the virtual environment as representations of a field-of-view (FOV) of the simulated sensor relative to the virtual environment, wherein the simulating comprises determining a number of the emissions that interact with the virtual environment including one or more simulated objects in the virtual environment; and

provide a representation of a capability of the sensor in the location and orientation simulated on the vehicle based on the behavior of the emissions by calculating a performance metric for the simulated sensor based on the behavior of the emissions, including a beam efficiency metric that defines a ratio of the number of emission interactions to a total number of simulated emissions to quantify wasted emissions that indicate which of the emissions did not interact with the one or more simulated objects.

Assignments (4)
CHANGE OF NAME Recorded Jun 22, 2023
From: WOVEN PLANET NORTH AMERICA, INC.
To: WOVEN BY TOYOTA, U.S., INC.
Reel/Frame 064065/0601 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2021
From: LYFT, INC.; BLUE VISION LABS UK LIMITED
To: WOVEN PLANET NORTH AMERICA, INC.
Reel/Frame 056927/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 12, 2021
From: LYFT, INC.; MAGNA AUTONOMOUS SYSTEMS, LLC
To: LYFT, INC.; MAGNA AUTONOMOUS SYSTEMS, LLC
Reel/Frame 057434/0623 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2019
From: DEVORE, DAVID BEARL; GRANIERI, ALEXANDER CHARLES; JONES, ALFRED CHARLES, II; XUE, HAN HAN
To: LYFT, INC.
Reel/Frame 050689/0991 →