Method and device for providing data for creating a digital map
A method for providing data for creating a digital map. The method includes: detecting surroundings sensor data of the surroundings during a measuring run of a physical system, preferably a vehicle, the surroundings sensor data capturing the surroundings in an at least partially overlapping manner, first surroundings sensor data including three-dimensional information, and second surroundings sensor data including two-dimensional information; extracting, with the aid of a first neural network situated in the physical system, at least one defined object from the first and second surroundings sensor data into first extracted data; and extracting, with the aid of a second neural network situated in the physical system, characteristic features including descriptors from the first extracted data into second extracted data, the descriptors being provided for a defined alignment of the second extracted data in a map creation process.
1. A method for providing data for creating a digital map, comprising the following steps:
obtaining surroundings sensor data captured in an overlapping manner during a measuring run of a physical system, the surroundings sensor data including:
first surroundings sensor data, which represents surroundings of the physical system as a first three-dimensional, cloud of points; and
second surroundings sensor data, which represents the surroundings of the physical system as a two-dimensional arrangement of pixels;
registering the first surroundings sensor data to the second surroundings sensor data so that each of the points of the cloud of the first surroundings sensor data is associated with a respective one of the pixels;
using a first neural network:
processing the first surroundings sensor data to identify a subset of the pixels of the second surroundings sensor data corresponding to a predefined object; and
based on the registration, selecting a subset of the points of the cloud of the first surroundings sensor data corresponding to the subset of the pixels, the selected subset of the points forming a three-dimensional representation of the predefined object;
extracting, using a second neural network:
a subset of the subset of the points corresponding to one or more characteristic features of the predefined object; and
descriptors of the one or more characteristic features;
based on the descriptors, aligning the subset of the subset of the points with a subset of another subset of points, which is of a second in three-dimensional cloud of points; and
based on the alignment of the subset of the subset of the points with the subset of the another subset of points, stitching the first and second three-dimensional clouds of points to each other, thereby creating at least a part of a three-dimensional map.
2. The method as recited in claim 1 , wherein the physical system is a vehicle.
3. The method as recited in claim 1 , wherein at least one of the following is used as the first surroundings sensor data: radar data, and/or LIDAR data, and/or stereo camera data, and wherein camera data is used as the second surroundings data.
4. The method as recited in claim 1 , wherein defined inaccuracy ranges of the surroundings sensor data are used as input data for the first and second neural networks.
5. The method as recited in claim 1 , wherein the characteristic features and the descriptors define details of the predefined object.
6. The method as recited in claim 5 , wherein the descriptors define a dimensional indication of the characteristic features.
7. The method as recited in claim 1 , wherein the first three-dimensional cloud of points and the second three-dimensional cloud of points are obtained from different runs of the physical system and/or at different points in time.
8. The method as recited in claim 1 , wherein the defined object is one of the following: a lane marking, a traffic sign, a lane, a zebra crossing, a road sign, and a building.
9. The method as recited in claim 1 , wherein the first neural network is a deep convolutional neural network and the second neural network is a point cloud-based neural network.
10. A device for creating a digital map, the device comprising:
a processor, wherein the processor is configured to:
obtain surroundings sensor data captured in an overlapping manner during a measuring run of a physical system, the surroundings sensor data including:
first surroundings sensor data, which represents surroundings of the physical system as a first three-dimensional cloud of points; and
second surroundings sensor data, which represents the surroundings of the physical system as a two-dimensional arrangement of pixels;
register the first surroundings sensor data to the second surroundings sensor data so that each of the points of the cloud of the first surroundings sensor data is associated with a respective one of the pixels;
use a first neural network to:
process the first surroundings sensor data to identify a subset of the pixels of the second surroundings sensor data corresponding to a predefined object; and
based on the registration, select a subset of the points of the cloud of the first surroundings sensor data corresponding to the subset of the pixels, the selected subset of the points forming a three-dimensional representation of the predefined object;
extract, using a second neural network:
a subset of the subset of the points corresponding to characteristic features of the predefined object; and
descriptors of the one or more characteristic features;
based on the descriptors, align the subset of the subset of the points with a subset of another subset of points, which is of a second three-dimensional cloud of points; and
based on the alignment of the subset of the subset of the points with the subset of the another subset of points, stitch the first and second three-dimensional clouds of points to each other, thereby creating at least a part of a three-dimensional map.
11. A non-transitory computer-readable storage medium on which is stored a computer program including program code for creating a digital map, the program code, when executed by a computer, causing the computer to perform the following steps:
obtaining surroundings sensor data captured in an overlapping manner during a measuring run of a physical system, the surroundings sensor data including:
first surroundings sensor data, which represents surroundings of the physical system as a first three-dimensional cloud of points; and
second surroundings sensor data, which represents the surroundings of the physical system as a two-dimensional arrangement of pixels;
registering the first surroundings sensor data to the second surroundings sensor data so that each of the points of the cloud of the first surroundings sensor data is associated with a respective one of the pixels;
using a first neural network:
processing the first surroundings sensor data to identify a subset of the pixels of the second surroundings sensor data corresponding to a predefined object; and
based on the registration, selecting a subset of the points of the cloud of the first surroundings sensor data corresponding to the subset of the pixels, the selected subset of the points forming a three-dimensional representation of the predefined object;
extracting, using a second neural network:
a subset of the subset of the points corresponding to one or more characteristic features of the predefined object; and
descriptors of the one or more characteristic features;
based on the descriptors, aligning the subset of the subset of the points with a subset of another subset of points, which is of a second three-dimensional cloud of points; and
based on the alignment of the subset of the subset of the points with the subset of the another subset of points, stitching the first and second three-dimensional clouds of points to each other, thereby creating at least a part of a three-dimensional map.