IP Library Granted Patent US 11,885,712
Granted Patent B2
US 11,885,712 · App. 17/093,172 · Granted Jan 30, 2024

Perception simulation for improved autonomous vehicle control

Inventors: Xing Sun (San Diego, CA); Wutu Lin (San Diego, CA); Yufei Zhao (San Diego, CA); Liu Liu (San Diego, CA)
Assignee: TUSIMPLE, INC.
G01M17/00B60W30/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,885,712
App. No.
17/093,172
Granted
Jan 30, 2024
Kind
B2
Abstract

A system and method for real world autonomous vehicle perception simulation are disclosed. A particular embodiment includes: configuring a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters; using the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and providing the simulated perception data to a motion planning system for the autonomous vehicle.

Claims (31)

1. A system comprising:

a data processor; and

a perception simulation module, executable by the data processor, the perception simulation module being configured to:

configure a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters;

use the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and

provide the simulated perception data to a motion planning system for an autonomous vehicle.

2. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating a false positive or false negative object detection.

3. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in vehicle localization.

4. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in lane detection.

5. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in object type classification.

6. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in object location.

7. The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in data produced by an image processing module.

8. A method comprising:

configuring a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters;

using the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and

providing the simulated perception data to a motion planning system for an autonomous vehicle.

9. The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating a false positive or false negative result produced by a perception processing module.

10. The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating noise produced by a perception sensor device.

11. The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating noise in perception sensor devices and sensor data processing modules over a pre-determined time period.

12. The method of claim 8 including producing simulated perception data that corresponds to actual or simulated ground truth data.

13. The method of claim 8 including configuring or training the motion planning system for the autonomous vehicle or a vehicle simulation system.

14. The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating errors corresponding to physical constraints of a perception sensor used with the autonomous vehicle.

15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

configure a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters;

use the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and

provide the simulated perception data to a motion planning system for an autonomous vehicle.

16. The non-transitory machine-useable storage medium of claim 15 wherein the plurality of sensors includes: image generating devices, cameras, distance measuring devices, light amplification by stimulated emission of radiation (laser) devices or light detection and ranging (LIDAR) devices, global positioning systems (GPS) devices, sound navigation and ranging (sonar) devices, or radio detection and ranging (radar) devices.

17. The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating a false positive or false negative object classification.

18. The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in vehicle trajectory.

19. The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in roadway detection.

20. The non-transitory machine-useable storage medium of claim 15 being configured to perform a calibration phase based on a comparison of real world perception data and ground truth data.

Assignments (3)
CHANGE OF NAME Recorded Dec 3, 2025
From: TUSIMPLE, INC.
To: CREATEAI, INC.
Reel/Frame 073832/0553 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2020
From: SUN, XING; LIN, WUTU; LIU, LIU; ZHAO, YUFEI
To: TUSIMPLE
Reel/Frame 054316/0898 →
CHANGE OF NAME Recorded Nov 9, 2020
From: TUSIMPLE
To: TUSIMPLE, INC.
Reel/Frame 054367/0675 →
Continuity (3)
Continuation 16660244 · Oct 22, 2019
Division 15598693 · May 18, 2017
Related Publication 20210080353A1 · Mar 18, 2021