Method, apparatus, and computer program product for map geometry generation based on data aggregation and conflation
A method is provided for automatically creating road and lane geometry from images representing probe data within a geographical area using data aggregation and conflation. Methods may include: receiving first observation data associated with a geographic area; processing the first observation data through a first encoder to produce first embeddings; receiving second observation data associated with the geographic area; processing the second observation data through a second encoder to produce second embeddings; establishing contrastive loss between the first embeddings and the second embeddings; producing consistent embeddings from the first embeddings and the second embeddings using the contrastive loss; generating, from the consistent embeddings, map data reflecting the first observation data and the second observation data; and updating a map database with the map data.
1 . An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the processor, cause the apparatus to at least:
receive first observation data associated with a geographic area, wherein the first observation data comprises sensor data from at least one sensor within the geographic area;
process the first observation data through a first encoder to produce first embeddings;
receive second observation data associated with the geographic area, wherein the second observation data comprises sensor data from at least one other sensor within the geographic area;
process the second observation data through a second encoder to produce second embeddings;
establish contrastive loss using a contrastive loss function between the first embeddings and the second embeddings;
produce consistent embeddings from the first embeddings and the second embeddings using the contrastive loss;
generate, from the consistent embeddings using a first map generator, map data reflecting the first observation data and the second observation data of point objects, and generate, from the consistent embeddings using a second map generator, map data reflecting the first observation data and the second observation data of line objects;
measure the map data reflecting the first observation data and the second observation data produced from the consistent embeddings against ground truth map data using a loss function to obtain a reconstruction loss;
train the first map generator and the second map generator using the reconstruction loss; and
update a map database with the map data of both the point objects and the line objects.
2 . The apparatus of claim 1 , wherein causing the apparatus to generate, from the consistent embeddings, map data reflecting the first observation data and the second observation data comprises causing the apparatus to:
generate, from the consistent embeddings, map data reflecting the first observation data and the second observation data;
determine a loss of the map data relative to ground truth data; and
update a map generator generating the map data using the loss.
3 . The apparatus of claim 1 , wherein the first observation data associated with the geographic area comprises at least one of aerial image data, probe data, LiDAR (Light Distancing and Ranging) data, or vehicle image sensor data, wherein the second observation data associated with the geographic area comprises at least one of aerial image data, probe data, LiDAR data, or vehicle image sensor data different from the first observation data.
4 . The apparatus of claim 3 , wherein causing the apparatus to establish contrastive loss between the first embeddings and the second embeddings comprises causing the apparatus to establish contrastive loss between two different types of data from two different sources of data.
5 . The apparatus of claim 1 , wherein the first encoder is a neural network that transforms the first observation data into the first embeddings, and the second encoder is a neural network that transforms the second observation data into the second embeddings.
6 . The apparatus of claim 1 , wherein causing the apparatus to produce consistent embeddings from the first embeddings and the second embeddings using the contrastive loss comprises causing the apparatus to:
align the first embeddings with the second embeddings based on the contrastive loss; and
generate the consistent embeddings based on the first observation data of a first data type and the second observation data of a second data type, different from the first data type.
7 . A method comprising:
receiving first observation data associated with a geographic area, wherein the first observation data comprises sensor data from at least one sensor within the geographic area;
processing the first observation data through a first encoder to produce first embeddings;
receiving second observation data associated with the geographic area, wherein the second observation data comprises sensor data from at least one other sensor within the geographic area;
processing the second observation data through a second encoder to produce second embeddings;
establishing contrastive loss using a contrastive loss function between the first embeddings and the second embeddings;
producing consistent embeddings from the first embeddings and the second embeddings using the contrastive loss;
generating, from the consistent embeddings using a first map generator, map data reflecting the first observation data and the second observation data of point objects, and generating, from the consistent embeddings using a second map generator, map data reflecting the first observation data and the second observation data of line objects;
measuring the map data reflecting the first observation data and the second observation data produced from the consistent embeddings against ground truth map data using a loss function to obtain a reconstruction loss;
training the first map generator and the second map generator using the reconstruction loss; and
updating a map database with the map data of both the point objects and the line objects.
8 . The method of claim 7 , wherein generating, from the consistent embeddings, map data reflecting the first observation data and the second observation data comprises:
generating, from the consistent embeddings, map data reflecting the first observation data and the second observation data;
determining a loss of the map data relative to ground truth data; and
updating a map generator generating the map data using the loss.
9 . The method of claim 7 , wherein the contrastive loss is determined using a contrastive language image pre-training (CLIP) loss function.
10 . The method of claim 7 , wherein the first observation data associated with the geographic area comprises at least one of aerial image data, probe data, LiDAR (Light Distancing and Ranging) data, or vehicle image sensor data, wherein the second observation data associated with the geographic area comprises at least one of aerial image data, probe data, LiDAR data, or vehicle image sensor data different from the first observation data.
11 . The method of claim 10 , wherein establishing contrastive loss between the first embeddings and the second embeddings comprises establishing contrastive loss between two different types of data from two different sources of data.
12 . The method of claim 7 , wherein the first encoder is a neural network that transforms the first observation data into the first embeddings, and the second encoder is a neural network that transforms the second observation data into the second embeddings.
13 . The method of claim 7 , wherein producing consistent embeddings from the first embeddings and the second embeddings using the contrastive loss comprises:
aligning the first embeddings with the second embeddings based on the contrastive loss; and
generating the consistent embeddings based on the first observation data of a first data type and the second observation data of a second data type, different from the first data type.
14 . The method of claim 7 , wherein the first observation data is received in at least near real-time and assembled into batches, wherein processing the first observation data through the first encoder to produce the first embeddings comprises processing a batch of the first observation data through the first encoder to produce the first embeddings.
15 . A system for generation of map data comprising:
at least one memory comprising a map database stored therein; and
processing circuitry configured to:
receive first observation data associated with a geographic area, wherein the first observation data comprises sensor data from at least one sensor within the geographic area;
process the first observation data through a first encoder to produce first embeddings;
receive second observation data associated with the geographic area, wherein the second observation data comprises sensor data from at least one other sensor within the geographic area;
process the second observation data through a second encoder to produce second embeddings;
establish contrastive loss using a contrastive loss function between the first embeddings and the second embeddings;
produce consistent embeddings from the first embeddings and the second embeddings using the contrastive loss;
generate, from the consistent embeddings using a first map generator, map data reflecting the first observation data and the second observation data of point objects, and generate, from the consistent embeddings using a second map generator, map data reflecting the first observation data and the second observation data of line objects;
measure the map data reflecting the first observation data and the second observation data produced from the consistent embeddings against ground truth map data using a loss function to obtain a reconstruction loss;
train the first map generator and the second map generator using the reconstruction loss; and
update the map database with the map data of both the point objects and the line objects.
16 . The system of claim 15 , wherein the processing circuitry configured to generate, from the consistent embeddings, map data reflecting the first observation data and the second observation data comprises processing circuitry configured to:
generate, from the consistent embeddings, map data reflecting the first observation data and the second observation data;
determine a loss of the map data relative to ground truth data; and
update a map generator generating the map data using the loss.
17 . The system of claim 15 , wherein the contrastive loss is determined using a contrastive language image pre-training (CLIP) loss function.
18 . The system of claim 15 , wherein the first observation data associated with the geographic area comprises at least one of aerial image data, probe data, LiDAR data, or vehicle image sensor data, wherein the second observation data associated with the geographic area comprises at least one of aerial image data, probe data, LiDAR data, or vehicle image sensor data different from the first observation data.
19 . The apparatus of claim 1 , wherein causing the apparatus to generate, from the consistent embeddings, map data reflecting the first observation data and the second observation data comprises causing the apparatus to generate map data independently from the map database.