Method and device for point of interest (POI) detection clustering using image embeddings
Aspects concern a method for point of interest (POI) detection clustering using image embeddings, the method including obtaining, using a neural network, the image embeddings from POI detections including portions of one or more images that are captured in an area of a map, in which at least one POI is detected, calculating an embedding distance between the obtained image embeddings, and recognizing texts included in the POI detections. The method further includes calculating an optical character recognition (OCR) distance between the recognized texts, calculating a final distance between the POI detections, based on the calculated embedding distance and the calculated OCR distance, and clustering the POI detections, based on the calculated final distance.
1 . A method for point of interest (POI) detection clustering using image embeddings, the method comprising:
obtaining, using a neural network, the image embeddings from filtered POI detections, the filtered POI detections comprising portions of one or more images that are captured in an area of a map, in which POIs are detected, wherein the filtered POI detections are obtained by calculating a coordinate distance between geographic coordinate pairs of each pair of the POI detections and removing, based on the coordinate distance, a respective pair of the POI detections from the POI detections;
calculating an embedding distance between the obtained image embeddings; recognizing texts included in the filtered POI detections;
calculating an optical character recognition (OCR) distance between the recognized texts;
calculating a final distance between the POI detections, based on the calculated embedding distance and the calculated OCR distance; and
clustering the POI detections, based on the calculated final distance;
updating the map to comprise a POI corresponding to the clustered POI detections; and
providing the updated map for generating control signals for a vehicle based on the updated map.
2 . The method of claim 1 , further comprising determining whether the calculated coordinate distance is greater than or equal to a predetermined threshold distance.
3 . The method of claim 2 , wherein the filtered POI detections includes removing the respective pair of the POI detections from the POI detections is performed based on the coordinate distance being determined to be greater than or equal to the predetermined threshold distance.
4 . The method of claim 3 , wherein the clustering the POI detections comprises clustering the filtered POI detections, based on the calculated final distance between the filtered POI detections.
5 . The method of claim 1 , wherein the embedding distance is a cosine distance between the obtained image embeddings, and is calculated based on a following equation:
similarity
=
cos
(
θ
)
=
A
·
B
A
B
=
∑
i
=
1
n
A
i
B
i
∑
i
=
1
n
A
i
2
∑
i
=
1
n
B
i
2
,
and
A and B denote the obtained image embeddings.
6 . The method of claim 1 wherein the calculating the OCR distance comprises removing common words from a pair of the recognized texts.
7 . The method of claim 6 , wherein the calculating the OCR distance further comprises calculating the OCR distance between an ordering of words in both of the pair of the recognized texts from which the common words are removed.
8 . The method of claim 1 , wherein the calculating the final distance comprises determining whether the calculated OCR distance is less than a first predetermined distance.
9 . The method of claim 8 , wherein the calculating the final distance further comprises determining whether the calculated embedding distance is less than a second predetermined distance.
10 . The method of claim 9 , wherein the calculating the final distance further comprises, based on the calculated OCR distance being determined to be less than the first predetermined distance and the calculated embedding distance being determined to be less than the second predetermined distance, setting the final distance to be zero.
11 . The method of claim 9 , wherein the calculating the final distance further comprises, based on the calculated OCR distance being determined to be greater than or equal to the first predetermined distance and/or the calculated embedding distance being determined to be greater than or equal to the second predetermined distance, setting the final distance to be the calculated embedding distance.
12 . The method of claim 1 , wherein the clustering the POI detections comprises determining whether the calculated final distance is less than a predefined threshold distance.
13 . The method of claim 12 , wherein the clustering the POI detections further comprises, based on the calculated final distance being determined to be less than the predefined threshold distance, clustering the POI detections.
14 . The method of claim 1 , further comprising:
providing the updated map to at least one of a user and a controller configured to generate control signals for a vehicle based on the updated map.
15 . A server configured to perform the method of claim 1 .
16 . A non-transitory computer-readable medium comprising program instructions, which, when executed by one or more processors, cause the one or more processors to perform the method of claim 1 .