AUTOMATICALLY GENERATING TRAINING DATA FOR A LIDAR USING SIMULATED VEHICLES IN VIRTUAL SPACE
Automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments are disclosed herein. The feature training datasets may be associated with training a machine learning model to control real-world autonomous vehicles. In some embodiments, an occupancy grid generator is used to generate an occupancy grid indicative of an environment of an autonomous vehicle from an imaging scene that depicts the environment. The occupancy grid is used to control the vehicle as the vehicle moves through the environment. In further embodiments, a sensor parameter optimizer may determine parameter settings for use by real-world sensors in autonomous driving applications. The sensor parameter optimizer may determine, based on operation of the autonomous vehicle, an optimal parameter setting of the parameter setting where the optimal parameter setting may be applied to a real-world sensor associated with real-world autonomous driving applications.
1 . A non-transitory computer-readable medium storing thereon instructions executable by one or more processors to implement an occupancy grid generator for generating an occupancy grid indicative of an environment of a vehicle from an imaging scene that depicts the environment, the occupancy grid generator comprising:
a normal layer component configured to generate a normal layer based on the imaging scene, the normal layer defining a two-dimensional (2D) view of the imaging scene,
a label layer component configured to generate a label layer, the label layer being mapped to the normal layer and encoded with a first channel set, and the first channel set being associated with one or more text-based or state-based values of one or more objects of the environment; and
a velocity layer component configured to generate a velocity layer, the velocity layer being mapped to the normal layer, the velocity layer being encoded with a second channel set, and the second channel set being associated with one or more velocity values of one or more objects of the environment,
wherein the occupancy grid generator generates an occupancy grid based on the normal layer, the label layer, and the velocity layer, the occupancy grid being used to control the vehicle as the vehicle moves through the environment.
2 . The non-transitory computer-readable medium of claim 1 , wherein the normal layer is a top-down graphical view of the virtual environment.
3 . The non-transitory computer-readable medium of claim 1 , wherein the first channel set includes a plurality of first channels of a pixel.
4 . The non-transitory computer-readable medium of claim 3 , wherein the plurality of first channels of the pixel include red (R), green (G), and blue (B) channels.
5 . The non-transitory computer-readable medium of claim 3 , wherein each of the plurality of first channels of the pixel indicates a particular text-based or state-based value.
6 . The non-transitory computer-readable medium of claim 1 , wherein the one or more text-based or state-based values define one or more classifications or one or more states of the one or more objects of the environment.
7 . The non-transitory computer-readable medium of claim 1 , wherein the second channel set includes a plurality of second channels of a pixel.
8 . The non-transitory computer-readable medium of claim 7 , wherein the plurality of second channels of the pixel include a red (R) channel, a green (G) channel, and a blue (B) channel.
9 . The non-transitory computer-readable medium of claim 8 , wherein the R channel defines a first component for the velocity layer, the G channel defines a second component for the velocity layer, and the B channel defines a third component for the velocity layer.
10 . The non-transitory computer-readable medium of claim 7 , wherein each of the plurality of second channels of the pixel indicates a particular velocity value.
11 . The non-transitory computer-readable medium of claim 1 , wherein the one or more velocity values define corresponding one or more velocities of one or more vehicles moving within the environment.
12 . The non-transitory computer-readable medium of claim 1 , wherein the occupancy grid generator further comprises a height layer component configured to generate a height layer, the height layer mapped to the normal layer, the height layer encoded with a third channel set, the third channel set associated with one or more height values.
13 . The non-transitory computer-readable medium of claim 12 , wherein the third channel set includes a plurality of third channels of a pixel.
14 . The non-transitory computer-readable medium of claim 13 , wherein the plurality of third channels of the pixel include red (R), green (G), and blue (B) channels.
15 . The non-transitory computer-readable medium of claim 13 , wherein each of the plurality of third channels of the pixel indicates a particular height value.
16 . The non-transitory computer-readable medium of claim 1 , wherein the imaging scene of the virtual environment is a frame in a set of frames, the set of frames defining the operation of the virtual vehicle within the virtual environment.
17 . The non-transitory computer-readable medium of claim 16 , wherein the set of frames form a video of the virtual vehicle operating in the virtual environment.
18 . The non-transitory computer-readable medium of claim 1 , wherein the environment is a virtual environment.
19 . The non-transitory computer-readable medium of claim 1 , wherein the environment is a real-world environment.
20 . An occupancy grid generation method for generating an occupancy grid indicative of an environment of a vehicle from an imaging scene that depicts the environment, the occupancy grid generation method comprising:
generating a normal layer based on the imaging scene, the normal layer defining a two-dimensional (2D) view of the imaging scene,
generating a label layer, the label layer being mapped to the normal layer and encoded with a first channel set, and the first channel set being associated with one or more text-based or state-based values of one or more objects of the environment;
generating a velocity layer, the velocity layer being mapped to the normal layer, the velocity layer being encoded with a second channel set, and the second channel set being associated with one or more velocity values of one or more objects of the environment; and
generating an occupancy grid based on the normal layer, the label layer, and the velocity layer, the occupancy grid being used to control the vehicle as the vehicle moves through the environment.