Training of a convolutional neural network
This disclosure is related to a method, a computer program code, and an apparatus for training a convolutional neural network for an autonomous driving system. The disclosure is further related to a convolutional neural network, to an autonomous driving system comprising a neural network, and to an autonomous or semi-autonomous vehicle comprising such an autonomous driving system. For training the convolutional neural network, in a first step real-world driving data are selected as training data. Furthermore, synthetic driving data are generated as training data. The convolutional neural network is then trained on the selected real-world driving data and the generated synthetic driving data using a genetic algorithm.
1 . A method for operating an autonomous driving system of an ego vehicle, the method comprising:
training a convolutional neural network including
selecting real-world driving data (X) as training data;
generating synthetic driving data ({circumflex over (X)}) as training data; and
training the convolutional neural network on the selected real-world driving data (X) and the generated synthetic driving data ({circumflex over (X)}) using a genetic algorithm;
selecting a driving strategy based on an output of the convolutional neural network; and
controlling steering of the ego vehicle using the driving strategy.
2 . The method according to claim 1 , wherein the training data are represented by paired sequences of occupancy grids and behavioral labels (Y, Ŷ).
3 . The method according to claim 2 , wherein the behavioral labels (Y, Ŷ) are composed of driving trajectories, steering angles and velocities.
4 . The method according to claim 3 , wherein the sequences of occupancy grids representing the real-world driving data (X) and the synthetic driving data ({circumflex over (X)}) are processed in parallel by a set of convolutional layers before being stacked.
5 . The method according to claim 4 , wherein the stacked processed occupancy grids are fed to an LSTM network via a fully connected layer.
6 . The method according to claim 5 , wherein the synthetic driving data ({circumflex over (X)}) are obtained using a generative process, which models the behavior of the ego vehicle and of other traffic participants.
7 . The method according to claim 6 , wherein the generative process uses a single-track kinematic model of a robot for generating artificial motion sequences of virtual agents.
8 . The method according to claim 7 , wherein variables controlling the behavior of the virtual agents are, for each virtual agent, the longitudinal velocity and the rate of change of the steering angle.
9 . An autonomous driving system configured to select a driving strategy, comprisng:
a convolutional neural network (CNN) that has been trained by:
selecting real-world driving data (X) as training data;
generating synthetic driving data ({circumflex over (X)}) as training data; and
training the convolutional neural network on the selected real-world- driving data (X) and the generated synthetic driving data ({circumflex over (X)}) using a genetic algorithm; and
a controller configured to output steering commands to control an ego vehicle using the driving strategy.
10 . The autonomous driving system according to claim 9 , wherein the training data are represented by paired sequences of occupancy grids and behavioral labels (Y, Ŷ).
11 . The autonomous driving system according to claim 10 , wherein the behavioral labels (Y, Ŷ) are composed of driving trajectories, steering angles and velocities.
12 . The autonomous driving system according to claim 11 , wherein the sequences of occupancy grids representing the real-world driving data (X) and the synthetic driving data ({circumflex over (X)}) are processed in parallel by a set of convolutional layers before being stacked.
13 . The autonomous driving system according to claim 12 , wherein the stacked processed occupancy grids are fed to an LSTM network via a fully connected layer.
14 . The autonomous driving system according to claim 13 , wherein the synthetic driving data ({circumflex over (X)}) are obtained using a generative process, which models the behavior of an ego vehicle and of other traffic participants.