Stop sign and traffic light alert
Embodiments relate generally to three-dimensional mapping and more particularly to creating and using three-dimensional feature maps to improve vehicle safety. Features of a road and surrounding area are identified from imaging and telemetry data. One or more constraints such as dimensions or distances of each feature are estimated from the telemetry and imaging data. A feature map is derived from the constraints and the identified features. Some example applications include using the feature map in automatic emergency braking systems or forward collision warning systems.
1. A method of three-dimensional mapping, comprising:
receiving telemetry data from a device, wherein the telemetry data corresponds to sensor data associated with the device;
receiving imaging data of a road and surrounding areas from the device;
identifying a set of features in the imaging data, the imaging data and the telemetry data comprising feature location information;
estimating, based on the imaging data and the telemetry data, a set of constraints related to each of the features identified in the imaging data;
generating a graph representing the set of features and the set of constraints, the graph comprising a set of vertices corresponding to the features and a set of edges corresponding to the constraints, the graph comprising at least one edge representing a distance between a first feature and a second feature and at least one edge representing a distance between the first feature and the device, the graph representing a non-linear system of equations, wherein each edge includes an estimated error associated with the corresponding constraint, the estimated error being a function of circumstances of an observation;
solving the non-linear system of equations to determine a plurality of feature locations, wherein the plurality of determined feature locations are more accurate than the feature location information from the imaging data and telemetry data;
generating, by a mapping platform, a three dimensional map of the set of features based on the determined feature locations.
2. The method of claim 1 , wherein the device is associated with a vehicle, and the vehicle is driving on the road.
3. The method of claim 2 , wherein a portion of the sensor data is generated by sensors attached to the vehicle.
4. The method of claim 2 , wherein the mapping platform is attached to the vehicle, and further comprising:
generating a compressed representation of the three-dimensional map by representing each feature as a labeled bounding box positioned in a three dimensional coordinate system; and
transmitting the compressed representation of the three dimensional map to a server remote from the vehicle.
5. The method of claim 4 , wherein the server receives the compressed representation of the three-dimensional map and merges it with a global three-dimensional map.
6. The method of claim 1 , wherein the set of constraints for each feature includes at least an estimate of a size of the feature and an estimate of a distance from the device to the feature.
7. The method of claim 6 , wherein the estimate of a size of the feature and the estimate of the distance from the device to the feature are determined from the imaging data using a monocular simultaneous location and mapping technique.
8. The method of claim 1 , wherein the identifying a set of features in the imaging data includes assigning each pixel of the imaging data a classification using a pre-trained classification network.
9. The method of claim 8 , wherein the pre-trained classification network is a convolutional neural network.
10. A method of localizing a vehicle on a road, comprising:
receiving, from a server, a global feature map identifying features of the road;
generating, by a mapping platform, a local feature map of the road and surrounding areas based on telemetry and imaging data acquired by sensors on the vehicle; and
identifying a location of the vehicle on the road based on a comparison between the global feature map and the local feature map.