AUTOMATICALLY GENERATING TRAINING DATA FOR A LIDAR USING SIMULATED VEHICLES IN VIRTUAL SPACE
Automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments are disclosed herein. The feature training datasets may be associated with training a machine learning model to control real-world autonomous vehicles. In some embodiments, an occupancy grid generator is used to generate an occupancy grid indicative of an environment of an autonomous vehicle from an imaging scene that depicts the environment. The occupancy grid is used to control the vehicle as the vehicle moves through the environment. In further embodiments, a sensor parameter optimizer may determine parameter settings for use by real-world sensors in autonomous driving applications. The sensor parameter optimizer may determine, based on operation of the autonomous vehicle, an optimal parameter setting of the parameter setting where the optimal parameter setting may be applied to a real-world sensor associated with real-world autonomous driving applications.
1 . A non-transitory computer-readable medium storing thereon instructions executable by one or more processors to implement a sensor parameter optimizer that determines parameter settings for use by real-world sensors in autonomous driving applications, the sensor parameter optimizer comprising:
an imaging engine configured to generate a plurality of imaging scenes defining a virtual environment;
a sensor simulator configured to receive a parameter setting for each of one or more virtual sensors, and generate, based on the parameter settings and the plurality of imaging scenes, sensor data indicative of current states of the virtual environment; and
an autonomous vehicle simulator configured to control an autonomous vehicle within the virtual environment based on the sensor data,
wherein the sensor parameter optimizer determines, based on operation of the autonomous vehicle, an optimal parameter setting of the parameter setting, the optimal parameter setting applied to a real-world sensor associated with real-world autonomous driving applications.
2 . The non-transitory computer-readable medium of claim 1 , wherein the parameter setting defines one or more of: a spatial distribution of scan lines of a point cloud, a field of regard, a range, or a location of a sensor associated with the autonomous vehicle.
3 . The non-transitory computer-readable medium of claim 1 , wherein the sensor simulator generates the sensor data via ray casting.
4 . The non-transitory computer-readable medium of claim 1 , wherein the sensor simulator generates simulated lidar or radar data.
5 . The non-transitory computer-readable medium of claim 1 , wherein the sensor simulates the sensor data using a graphic shader.
6 . The non-transitory computer-readable medium of claim 1 , wherein a particular object or surface is associated with a reflectivity value within the virtual environment, and wherein the sensor simulator generates at least a portion of the sensor data based on the reflectivity value.
7 . The non-transitory computer-readable medium of claim 6 , wherein the reflectivity value is derived from a color of the particular object or surface.
8 . The non-transitory computer-readable medium of claim 1 , wherein the sensor data is accessed via direct memory access (DMA).
9 . The non-transitory computer-readable medium of claim 1 , wherein the parameter setting is a user-configured parameter setting.
10 . The non-transitory computer-readable medium of claim 1 , wherein the optimal parameter setting is based on evolutionary learning, the evolutionary learning based on vehicle operation data captured when the autonomous vehicle interacts with one or more objects or surfaces with the virtual environment.
11 . The non-transitory computer-readable medium of claim 10 , wherein the evolutionary learning is reinforcement learning.
12 . A sensor parameter optimizer method for determining parameter settings for use by real-world sensors in autonomous driving applications, the sensor parameter optimizer method comprising:
generating a plurality of imaging scenes defining a virtual environment;
receiving a parameter setting for each of one or more virtual sensors, and generate, based on the parameter settings and the plurality of imaging scenes, sensor data indicative of current states of the virtual environment;
controlling an autonomous vehicle within the virtual environment based on the sensor data; and
determining, based on operation of the autonomous vehicle, an optimal parameter setting of the parameter setting, the optimal parameter setting applied to a real-world sensor associated with real-world autonomous driving applications.