IP Library › Granted Patent US 12,638,303
Granted Patent B2
US 12,638,303 · App. 17/515,730 · Granted May 26, 2026

Apparatus, method, and computer program for generating map

Inventor: Mamoru Tsukamoto (Bunkyo-ku, JP)
Assignee: TOYOTA JIDOSHA KABUSHIKI KAISHA
G01C21/3822G01C21/32G01C21/3852G01C21/3867
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,638,303
App. No.
17/515,730
Filed
Nov 1, 2021
Granted
May 26, 2026
Kind
B2
Art Unit
3667
USPC
701/532
Abstract

An apparatus for generating a map includes a processor configured to detect road areas representing roads from a bird's-eye view image, detect skeletal lines of the road areas and detects individual branch points of the skeletal lines as candidates for the centers of intersections, detect candidate intersection areas for the respective candidates for the centers of intersections from the bird's-eye view image, and detect two or more of the candidate intersection areas that at least partially overlap each other as a single intersection area. The candidate intersection areas respectively include the candidates for the centers.

Claims (55)

1 . An apparatus for generating a map, comprising:

a processor configured to:

determine one or more stop lines by at least providing a bird's-eye view image to a machine learning classifier trained to detect stop lines,

detect road areas representing roads from the bird's-eye view image,

detect skeletal lines of the road areas and detect individual branch points of the skeletal lines as candidates for the centers of intersections,

detect candidate intersection areas for candidates for the centers of intersections from the bird's-eye view image based on the one or more stop lines, wherein an outer boundary of a candidate intersection area is determined by extending scan lines radially from a candidate for a center of an intersection, checking, for each of the scan lines, for (i) a border of a road area and (ii) an extension of a stop line to a road edge of the road area, and setting, for each of the scan lines, an outer edge of the candidate intersection area at a position where the scan line crosses the border or the extension,

detect two or more of candidate intersection areas that at least partially overlap each other as a single combined intersection area,

detect, for each candidate intersection area, and for each road connected to the intersection area, a number and a position of entry lanes and a number and position of exit lanes of each road, based on a length and a position of a stop line associated with each road, wherein each candidate intersection area contains at least one entry lane and at least one exit lane,

provide the candidate intersection areas to a variational autoencoder,

generate a lane network by the variational autoencoder, wherein the lane network connects, for each road connected to each candidate intersection area, each entry lane of the road to at least one exit lane of the at least one exit lane, and

generate a first road map using at least the lane network.

2 . The apparatus according to claim 1 , wherein the processor is further configured to detect a road marking formed across a road from the bird's-eye view image, and

the processor detects the two or more of the candidate intersection areas that at least partially overlap each other as separate intersection areas when the road marking exists between the candidates for the centers of the two or more of the candidate intersection areas.

3 . The apparatus according to claim 1 , wherein the processor is further configured to

detect, from among the candidates for the centers of intersections, a plurality of candidates connected by the skeletal lines to form a closed curve,

determine whether the closed curve is approximated by a circle centered at a centroid of the plurality of candidates, and

detect an area including the detected plurality of candidates as an area representing a rotary when the closed curve is approximated by a the circle.

4 . The apparatus according to claim 1 , wherein the processor is further configured to detect a road edge or a road marking indicating a road edge from the bird's-eye view image, and

the processor does not detect one of the candidate intersection areas corresponding to one of the candidates for the centers of intersections as the intersection area when one of the skeletal lines passing through the one of the candidates for the centers crosses the road edge or the road marking indicating a road edge along another of the skeletal lines passing through the one of the candidates for the centers.

5 . The apparatus according to claim 1 , wherein the bird's-eye view image includes altitude information indicating real-space altitudes of locations represented in respective pixels of the bird's-eye view image, and

the processor does not detect one of the candidate intersection areas corresponding to one of the candidates for the centers of intersections as the intersection area when a difference between a real-space altitude of a location corresponding to a position on one of the skeletal lines passing through the one of the candidates for the centers and a real-space altitude of a location corresponding to a position on another of the skeletal lines passing through the one of the candidates for the centers is not less than a predetermined altitude difference threshold.

6 . The apparatus according to claim 4 , wherein the detecting the road edge or the road marking includes extending the scan lines such that the scan lines are drawn radially from each candidate intersection area at equiangular intervals.

7 . The apparatus according to claim 1 , wherein the processor is further configured to:

divide each road of the first road map into first segments; and

determine geographical positions of the first segments based on information indicating the geographical area represented in the bird's-eye view image and positions of the first segments in the bird's-eye view image.

8 . The apparatus according to claim 1 , wherein the processor is further configured to:

generate a second road map based on a second bird's-eye view image; and

generate a third road map based on connecting the first road map to the second road map such that a position of a road included in the first road map matches the position of the road in the second road map.

9 . The apparatus according to claim 8 , wherein connecting the first road map to the second road map is based on information indicating the geographical areas represented in the first bird's-eye image and the second bird's-eye image.

10 . The apparatus according to claim 1 , wherein there are at least four roads connected to at least one of the candidate intersection areas.

11 . The apparatus according to claim 1 , wherein there are at least four roads connected to the combined intersection area.

12 . The apparatus according to claim 1 , wherein the combined intersection area includes at least a plurality of nodes of orders not less than three.

13 . The apparatus according to claim 12 , wherein the combined intersection area has a low degree of symmetry.

14 . The apparatus according to claim 1 , wherein generating the lane network further includes generating, by the variational autoencoder, and using values of pixels in each candidate intersection area, a feature map.

15 . The apparatus according to claim 1 , wherein generating the lane network further includes providing the generated feature map to a graph parsing neural network.

16 . A method for generating a map, comprising:

determining one or more stop lines by at least providing a bird's-eye view image to a machine learning classifier trained to detect stop lines;

detecting road areas representing roads from a bird's-eye view image;

detecting skeletal lines of the road areas and detecting individual branch points of the skeletal lines as candidates for the centers of intersections;

detecting candidate intersection areas for the respective candidates for the centers of intersections from the bird's-eye view image based on the one or more stop lines, wherein an outer boundary of a candidate intersection area is determined by extending scan lines radially from a candidate for a center of an intersection, checking, for each of the scan lines, for (i) a border of a road area and (ii) an extension of a stop line to a road edge of the road area, and setting, for each of the scan lines, an outer edge of the candidate intersection area at a position where the scan line crosses the border or the extension;

detecting two or more of candidate intersection areas that at least partially overlap each other as a single combined intersection area;

detecting, for each candidate intersection area, and for each road connected to the intersection area, a number and a position of entry lanes and a number and position of exit lanes of each road, based on a length and a position of a stop line associated with each road, wherein each candidate intersection area contains at least one entry lane and at least one exit lane;

providing the candidate intersection areas to a variational autoencoder,

generating a lane network by the variational autoencoder, wherein the lane network connects, for each road connected to each candidate intersection area, each entry lane of the road to at least one exit lane of the at least one exit lane; and

generating a first road map using at least the lane network.

17 . A non-transitory recording medium that stores a computer program for generating a map, the computer program causing a computer to execute a process comprising:

determining one or more stop lines by at least providing a bird's-eye view image to a machine learning classifier trained to detect stop lines;

detecting road areas representing roads from a bird's-eye view image;

detecting skeletal lines of the road areas and detecting individual branch points of the skeletal lines as candidates for the centers of intersections;

detecting candidate intersection areas for the respective candidates for the centers of intersections from the bird's-eye view image based on the one or more stop lines, wherein an outer boundary of a candidate intersection area is determined by extending scan lines radially from a candidate for a center of an intersection, checking, for each of the scan lines, for (i) a border of a road area and (ii) an extension of a stop line to a road edge of the road area, and setting, for each of the scan lines, an outer edge of the candidate intersection area at a position where the scan line crosses the border or the extension;

detecting two or more of candidate intersection areas that at least partially overlap each other as a single combined intersection area;

detecting, for each candidate intersection area, and for each road connected to the intersection area, a number and a position of entry lanes and a number and position of exit lanes of each road, based on a length and a position of a stop line associated with each road, wherein each candidate intersection area contains at least one entry lane and at least one exit lane;

providing the candidate intersection areas to a variational autoencoder;

generating a lane network by the variational autoencoder, wherein the lane network connects, each entry lane of the road to at least one exit lane; and

generating a first road map using at least the lane network.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2021
From: TSUKAMOTO, MAMORU
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 057979/0959 →
Priority Claims (1)
JP 2020-190180 · Nov 16, 2020 · national
Continuity (1)
Related Publication 20220155097A1 · May 19, 2022
References Cited (40)
US 10664709B2 · You · 2020 [cited by applicant]
US 11195418B1 · Hong · 2021 [cited by examiner]
US 11537134B1 · Wiest · 2022 [cited by examiner]
US 11555706B1 · Levihn · 2023 [cited by examiner]
US 20040210383A1 · Sato · 2004 [cited by applicant]
US 20060217879A1 · Ikeuchi · 2006 [cited by examiner]
US 20070014488A1 · Chen · 2007 [cited by examiner]
US 20130328924A1 · Arikan · 2013 [cited by examiner]
US 20160102986A1 · Ma · 2016 [cited by examiner]
US 20190205640A1 · Zhan · 2019 [cited by examiner]
US 20200073977A1 · Montemerlo · 2020 [cited by examiner]
US 20200124725A1 · Qiu · 2020 [cited by examiner]
US 20200134354A1 · Viswanathan · 2020 [cited by examiner]
US 20200164872A1 · Yanagihara · 2020 [cited by examiner]
US 20200166371A1 · Fitchett · 2020 [cited by examiner]
US 20200200542A1 · Viswanathan · 2020 [cited by examiner]
US 20200226175A1 · Codevilla · 2020 [cited by examiner]
US 20200256698A1 · Shi · 2020 [cited by examiner]
US 20200278210A1 · Nomura · 2020 [cited by examiner]
US 20200286256A1 · Houts · 2020 [cited by examiner]
US 20200327707A1 · Furger · 2020 [cited by examiner]
US 20210302960A1 · McGill, Jr. · 2021 [cited by examiner]
US 20210342586A1 · Fleisig · 2021 [cited by examiner]
US 20220412770A1 · Li · 2022 [cited by examiner]
CN 107953882A · 2018 [cited by applicant]
JP 200365781A · 2003 [cited by applicant]
JP 201026326A · 2010 [cited by applicant]
JP 2010236996A · 2010 [cited by applicant]
JP 2010276553A · 2010 [cited by applicant]
JP 2017054346A · 2017 [cited by applicant]
JP 2019101256A · 2019 [cited by applicant]
JP 2019121260A · 2019 [cited by applicant]
JP 20208681A · 2020 [cited by applicant]
JP 2020038365A · 2020 [cited by applicant]
Itonaga, W., Matsuda, I., Yoneyama, N. and Ito, S. (2003), Automatic extraction of road networks from map images. Electron. Comm. Jpn. Pt. II, 86: 62-72. https://doi.org/10.1002/ecjb.10144 (Year: 2003). [cited by examiner]
J. Dai, Y. Wang, W. Li and Y. Zuo, “Automatic Method for Extraction of Complex Road Intersection Points From High-Resolution Remote Sensing Images Based on Fuzzy Inference, ” in IEEE Access, vol. 8, pp. 39212-39224, 202… [cited by examiner]
Ying Wang, Qinfen Zheng, Recognition of roads and bridges in SAR images, Pattern Recognition, vol. 31, Issue 7, 1998, pp. 953-962, ISSN 0031-3203, https://doi.org/10.1016/S0031-3203(97)00098-8. (Year: 1998). [cited by examiner]
J. Yuan, G. Lu, P. Friedrich and Y. Xu, “Real-time long-range road estimation in unknown environments,” 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA, 2020, pp. 83-88, doi: 10.1109/IV47402.2020.93047… [cited by examiner]
A. Plebe and M. D. Lio, “On the Road With 16 Neurons: Towards Interpretable and Manipulable Latent Representations for Visual Predictions in Driving Scenarios, ” in IEEE Access, vol. 8, pp. 179716-179734, 2020, doi: 10.… [cited by examiner]
C. Ye, J. Li, H. Jiang, H. Zhao, L. Ma and M. Chapman, “Semi-Automated Generation of Road Transition Lines Using Mobile Laser Scanning Data,” in IEEE Transactions on Intelligent Transportation Systems, vol. 21, No. 5, p… [cited by examiner]