IP Library Granted Patent US 12,399,033
Granted Patent B2
US 12,399,033 · App. 17/652,953 · Granted Aug 26, 2025

Method and apparatus for generating maps from aligned geospatial observations

Inventor: Tero Juhani Keski-Valkama (Zurich, CH)
Assignee: HERE GLOBAL B.V.
G01C21/3867B60W60/001G01C21/3815G01C21/3841G01C21/3878B60W2556/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,399,033
App. No.
17/652,953
Granted
Aug 26, 2025
Kind
B2
Abstract

A method, apparatus and computer program product are provided for learning to generate maps from raw geospatial observations from sensors traveling within an environment. Methods may include: receiving a plurality of sequences of geospatial observations from discrete trajectories; aligning the discrete trajectories generating aligned geospatial observations; concatenating the aligned geospatial observations; performing attentional clustering on the concatenated, aligned geospatial observations to obtain a set of entities with feature dimensionality; processing the set of entities through an iterative attention model incorporating a Gated Recurrent Unit gating pattern to obtain attentional layer outputs; generating, from one or more Set Transformers, a feature set of map object geometries based, at least in part, on the attentional layer outputs; updating a map geometry based on the feature set from the Set Transformers generating an updated map geometry; and provide for navigational assistance or at least semi-autonomous vehicle control based on the updated map geometry.

Claims (47)

1. An apparatus comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least:

receive a plurality of sequences of geospatial observations from discrete trajectories;

align the discrete trajectories to generate aligned geospatial observations;

concatenate the aligned geospatial observations;

perform attentional clustering on the concatenated, aligned geospatial observations to obtain a set of entities with feature dimensionality;

process the set of entities through an iterative attentional model incorporating a Gated Recurrent Unit (GRU) gating pattern to obtain attentional layer outputs, wherein the gating pattern is equivalent to multiple GRU gating modules in parallel, each processing its own stream of values as multiplexed by an attentional layer based on an inducing vector that iteratively updates;

generate, from one or more Set Transformers, a feature set of map object geometries based, at least in part, on the attentional layer outputs;

update a map geometry based on the feature set from each of the one or more Set Transformers to form an updated map geometry;

update a map database to include the map geometry including the map object geometries from the geospatial observations; and

provide for at least one of navigational assistance or at least semi-autonomous vehicle control based on the updated map geometry.

2. The apparatus according to claim 1 , wherein each of the one or more Set Transformers correspond to an object type, wherein the feature set from each of the one or more Set Transformers comprises a set of map object geometries of the object type.

3. The apparatus according to claim 1 , wherein the feature set comprises location information associated with a respective map object geometry.

4. The apparatus according to claim 1 , wherein the map object geometries comprise point objects and linear objects, wherein point objects comprise at least one of signs or poles, and wherein linear objects comprise at least one of road markings or road boundaries.

5. The apparatus according to claim 4 , wherein the point objects and linear objects from the geospatial observations are used to facilitate autonomous vehicle control.

6. The apparatus according to claim 1 , wherein causing the apparatus to align the discrete trajectories to generate the aligned geospatial observations comprises causing the apparatus to apply a drive offset to each of the discrete trajectories to generate the aligned geospatial observations.

7. The apparatus according to claim 1 , wherein the plurality of sequences of geospatial observations from discrete trajectories include unique trajectory identifiers for each discrete trajectory, wherein causing the apparatus to concatenate the aligned geospatial observations comprises causing the apparatus to remove the unique trajectory identifiers associated with the geospatial observations.

8. A method comprising:

receiving a plurality of sequences of geospatial observations from discrete trajectories;

aligning the discrete trajectories to generate aligned geospatial observations;

concatenating the aligned geospatial observations;

performing attentional clustering on the concatenated, aligned geospatial observations to obtain a set of entities with feature dimensionality;

processing the set of entities through an iterative attentional model incorporating a Gated Recurrent Unit (GRU) gating pattern to obtain attentional layer outputs, wherein the gating pattern is equivalent to multiple GRU gating modules in parallel, each processing its own stream of values as multiplexed by an attentional layer based on an inducing vector that iteratively updates;

generating, from one or more Set Transformers, a feature set of map object geometries based, at least in part, on the attentional layer outputs;

updating a map geometry based on the feature set from each of the one or more Set Transformers to form an updated map geometry;

updating a map database to include the map geometry including the map object geometries from the geospatial observations; and

providing for at least one of navigational assistance or at least semi-autonomous vehicle control based on the updated map geometry.

9. The method according to claim 8 , wherein each of the one or more Set Transformers correspond to an object type, wherein the feature set from each of the one or more Set Transformers comprises a set of map object geometries of the object type.

10. The method according to claim 8 , wherein the feature set comprises location information associated with a respective map object geometry.

11. The method according to claim 8 , wherein the map object geometries comprise point objects and linear objects, wherein point objects comprise at least one of signs or poles, and wherein linear objects comprise at least one of road markings or road boundaries.

12. The method according to claim 11 , wherein the point objects and linear objects from the geospatial observations are used to facilitate autonomous vehicle control.

13. The method according to claim 8 , wherein aligning the discrete trajectories to generate the aligned geospatial observations comprises applying a drive offset to each of the discrete trajectories to generate the aligned geospatial observations.

14. The method according to claim 8 , wherein the plurality of sequences of geospatial observations from discrete trajectories include unique trajectory identifiers for each discrete trajectory, wherein concatenating the aligned geospatial observations comprises causing the apparatus to remove removing the unique trajectory identifiers associated with the geospatial observations.

15. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:

receive a plurality of sequences of geospatial observations from discrete trajectories;

align the discrete trajectories to generate aligned geospatial observations;

concatenate the aligned geospatial observations;

perform attentional clustering on the concatenated, aligned geospatial observations to obtain a set of entities with feature dimensionality;

process the set of entities through an iterative attentional model incorporating a Gated Recurrent Unit (GRU) gating pattern to obtain attentional layer outputs, wherein the gating pattern is equivalent to multiple GRU gating modules in parallel, each processing its own stream of values as multiplexed by an attentional layer based on an inducing vector that iteratively updates;

generate, from one or more Set Transformers, a feature set of map object geometries based, at least in part, on the attentional layer outputs;

update a map geometry based on the feature set from each of the one or more Set Transformers to form an updated map geometry;

update a map database to include the map geometry including the map object geometries from the geospatial observations; and

provide for at least one of navigational assistance or at least semi-autonomous vehicle control based on the updated map geometry.

16. The computer program product according to claim 15 , wherein each of the one or more Set Transformers correspond to an object type, wherein the feature set from each of the one or more Set Transformers comprises a set of map object geometries of the object type.

17. The computer program product according to claim 15 , wherein the feature set comprises location information associated with a respective map object geometry.

18. The computer program product according to claim 15 , wherein the map object geometries comprise point objects and linear objects, wherein point objects comprise at least one of signs or poles, and wherein linear objects comprise at least one of road markings or road boundaries.

19. The computer program product according to claim 18 , wherein the point objects and linear objects from the geospatial observations are used to facilitate autonomous vehicle control.

20. The computer program product according to claim 15 , wherein the program code instructions to align the discrete trajectories to generate the aligned geospatial observations comprise program code instructions to apply a drive offset to each of the discrete trajectories to generate the aligned geospatial observations.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 3, 2022
From: KESKI-VALKAMA, TERO JUHANI
To: HERE GLOBAL B.V.
Reel/Frame 059317/0601 →
Continuity (1)
Related Publication 20230280186A1 · Sep 7, 2023
References Cited (47)
US 10969232B1 · Johnson et al. · 2021 [cited by applicant]
US 20180023961A1 · Fridman · 2018 [cited by applicant]
US 20180188027A1 · Zhang et al. · 2018 [cited by applicant]
US 20180188039A1 · Chen et al. · 2018 [cited by applicant]
US 20180190016A1 · Yang · 2018 [cited by examiner]
US 20180194286A1 · Stein · 2018 [cited by applicant]
US 20190384304A1 · Towal et al. · 2019 [cited by applicant]
US 20200173805A1 · Viswanathan · 2020 [cited by applicant]
US 20210063200A1 · Kroepfl · 2021 [cited by examiner]
US 20210191395A1 · Gao et al. · 2021 [cited by applicant]
US 20210248460A1 · Sykora et al. · 2021 [cited by applicant]
US 20210302992A1 · Chen et al. · 2021 [cited by applicant]
US 20220011122A1 · He · 2022 [cited by examiner]
US 20220153310A1 · Yang et al. · 2022 [cited by applicant]
US 20220236077A1 · Clarysse et al. · 2022 [cited by applicant]
US 20220237682A1 · Zhao · 2022 [cited by examiner]
US 20220277472A1 · Birchfield · 2022 [cited by examiner]
US 20220297728A1 · Varadarajan et al. · 2022 [cited by applicant]
US 20230050402A1 · Keski-Valkama · 2023 [cited by examiner]
US 20230100413A1 · Zhu · 2023 [cited by examiner]
US 20230245373A1 · Xiong et al. · 2023 [cited by applicant]
US 20230252224A1 · Tran · 2023 [cited by examiner]
US 20230260271A1 · Creswell · 2023 [cited by examiner]
CN 112883200A · 2021 [cited by applicant]
WO WO2021139283A1 · 2021 [cited by applicant]
F. Xue, X. Wang, J. Wang and H. Zha, “Deep Visual Odometry With Adaptive Memory,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, No. 2, pp. 940-954, Feb. 1, 2022, doi: 10.1109/TPAMI.2020.301… [cited by examiner]
Jaegle et al., “Perceiver: General Perception with Iterative Attention”, arXiv:2103.03206v2, (Jun. 23, 2021), 43 pages. [cited by applicant]
“Wikipedia—Gated Recurrent Unit”, Retrieved on Jul. 21, 2022, Retrieved via the Internet: <URL:https://en.wikipedia.org/w/index.php?title=Gated_recurrent_unit&oldid=107397 9539>, (Feb. 25, 2022), 5 pages. [cited by applicant]
Mobiny et al, “Trans-Caps: Transformer Capsule Networks with Self-attention Routing”, International Conference on Learning Representations 2021, (Sep. 28, 2020), 13 pages. [cited by applicant]
Jiang et al., “GAL: Geometric Adversarial Loss for Single-View 3D-Object Reconstruction”, European Conference on Computer Vision 2018, (Oct. 7, 2018), 15 pages. [cited by applicant]
Henderson, J., “The Unstoppable Rise of Computational Linguistics in Deep Learning”, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, (Jul. 2020), 13 pages. [cited by applicant]
Prakash, J., “Non Maximum Suppression: Theory and Implementation in PyTorch”, Retrieved on Jul. 13, 2022, Retrieved via the Internet: <URL:https://learnopencv.com/non-maximum-suppression-theory-and-implementation-in-pyt… [cited by applicant]
U.S. Appl. No. 17/444,861, filed Aug. 11, 2021, entitled “Method and Apparatus for Generating Maps from Geospatial Observations”, 39 pages. [cited by applicant]
U.S. Appl. No. 17/652,950, filed Mar. 1, 2022, entitled “Method, Apparatus, And Computer Program Product For Differential Variable Count Entity Output Head For Map Generation”, 51 pages. [cited by applicant]
U.S. Appl. No. 17/652,951, filed Mar. 1, 2022, entitled “Method and Apparatus for Generating Maps from Aligned Geospatial Observations”, 53 pages. [cited by applicant]
U.S. Appl. No. 17/652,952, filed Mar. 1, 2022, entitled “Method and Apparatus for Generating Maps from Aligned Geospatial Observations”, 53 pages. [cited by applicant]
Girgis et al., “Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction”, arXiv:2104.00563v3, (Feb. 11, 2022), 26 pages. [cited by applicant]
Hunter et al., “The Path Inference Filter: Model-Based Low-Latency Map Matching of Probe Vehicle Data”, IEEE Transactions on Intelligent Transportation Systems, vol. 15, No. 2, (Apr. 2014), 23 pages. [cited by applicant]
Lee et al., “Set Transformer: A Framework for Attention-Based Permutation-Invariant Neural Networks”, Proceedings of the 36th International Conference on Machine Learning, (May 24, 2019), 10 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/652,951 dated Mar. 21, 2024. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/652,952 dated Jun. 12, 2024. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/652,950 dated Apr. 1, 2024. [cited by applicant]
Final Office Action for U.S. Appl. No. 17/652,951 dated Feb. 7, 2025. [cited by applicant]
Final Office Action for U.S. Appl. No. 17/652,952 dated Dec. 12, 2024. [cited by applicant]
Advisory Action for U.S. Appl. No. 17/652,951 dated Apr. 15, 2025. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/652,952 dated Mar. 28, 2025. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/652,951 dated Jun. 26, 2025. [cited by applicant]