REALISTIC 3D VIRTUAL WORLD CREATION AND SIMULATION FOR TRAINING AUTOMATED DRIVING SYSTEMS
A computer implemented method of creating a simulated realistic virtual model of a geographical area for training an autonomous driving system, comprising obtaining geographic map data of a geographical area, obtaining visual imagery data of the geographical area, classifying static objects identified in the visual imagery data to corresponding labels to designate labeled objects, superimposing the labeled objects over the geographic map data, generating a virtual 3D realistic model emulating the geographical area by synthesizing a corresponding visual texture for each of the labeled objects and injecting synthetic 3D imaging feed of the realistic model to imaging sensor(s) input(s) of the autonomous driving system controlling movement of an emulated vehicle in the realistic model where the synthetic 3D imaging feed is generated to depict the realistic model from a point of view of emulated imaging sensor(s) mounted on the emulated vehicle.
1 . A system for controlling a driving system, comprising:
a data processor configured to:
obtain a sequence of images captured while traveling in a geographic area,
classify a plurality of objects identified in the sequence of images to corresponding labels using a neural network,
superimpose the plurality of labeled objects over geographic map data obtained for the geographic area, to create a labeled model of the geographical area,
create, using the labeled model, an output feed for a driving system, and
inject the output feed into the driving system.
2 . The system of claim 1 , wherein the neural network is a Convolutional Neural Network (CNN).
3 . The system of claim 1 , wherein the neural network is a Conditional Generative Adversarial Neural Network (cGAN).
4 . The system of claim 1 , wherein the data processor is further configured to remove objects from the output feed.
5 . The system of claim 1 , wherein the data processor is further configured to insert objects into the output feed.
6 . The system of claim 1 , wherein a software interface is used for injecting the output feed into the driving system.
7 . The system of claim 1 , wherein the output feed is generated using a virtual realistic model of the geographic area.
8 . The system of claim 7 , wherein the data processor is further configured to:
obtain geographic map data of the geographic area;
obtain visual imagery data of the geographic area;
classify a plurality of static objects identified in the visual imagery data to corresponding labels using the neural network; and
generate a virtual three dimensional (3D) realistic model using the corresponding labels of one or more of the plurality of labeled static objects.
9 . The system of claim 1 , wherein the driving system controls an autonomous vehicle.
10 . A method for controlling a driving system, comprising:
obtaining a sequence of images captured while traveling in a geographic area,
classifying a plurality of objects identified in the sequence of images to corresponding labels using a neural network,
superimposing the plurality of labeled objects over geographic map data obtained for the geographic area, to create a labeled model of the geographical area,
creating, using the labeled model, an output feed for a driving system, and
injecting the output feed into the driving system.
11 . The method of claim 10 , wherein the neural network is a Convolutional Neural Network (CNN).
12 . The method of claim 10 , wherein the neural network is a Conditional Generative Adversarial Neural Network (cGAN).
13 . The method of claim 10 , wherein the data processor is further configured to remove objects from the output feed.
14 . The method of claim 10 , wherein the data processor is further configured to insert objects into the output feed.
15 . The method of claim 10 , wherein a software interface is used for injecting the output feed into the driving system.
16 . The method of claim 10 , wherein the output feed is created using a virtual realistic model of the geographic area.
17 . The method of claim 16 , wherein the data processor is further configured to:
obtain geographic map data of the geographic area;
obtain visual imagery data of the geographic area;
classify a plurality of static objects identified in the visual imagery data to corresponding labels using the neural network; and
create a virtual three dimensional (3D) realistic model using the corresponding labels of one or more of the plurality of labeled static objects.
18 . The method of claim 10 , wherein the driving system controls an autonomous vehicle.
19 . A computer program product comprising:
a non-transitory computer readable storage medium;
program instructions which, when executed by a processor, cause the processor performing operations for controlling a driving system, the operations comprising:
obtaining a sequence of images captured while traveling in a geographic area,
classifying a plurality of objects identified in the sequence of images to corresponding labels using a neural network,
superimposing the plurality of labeled objects over geographic map data obtained for the geographic area, to create a labeled model of the geographical area,
creating, using the labeled model, an output feed for a driving system, and
injecting the output feed into the driving system.