IP Library › Granted Patent US 12,039,788
Granted Patent B2
US 12,039,788 · App. 17/584,879 · Granted Jul 16, 2024

Path planning method and system using the same

Inventors: Jiun-In Guo (Hsinchu, TW); Jen-Shuo Chang (Taichung, TW)
Assignee: National Yang Ming Chiao Tung University
G06V20/588G01S17/89G06T7/143G06V10/763G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,039,788
App. No.
17/584,879
Granted
Jul 16, 2024
Kind
B2
Abstract

A path planning system includes an image-capturing device, a point-cloud map-retrieving device, and a processing device. The image-capturing device captures a first and a second camera road image. The point-cloud map-retrieving device retrieves distance data points to create a road distance point-cloud map. The processing device receives the road distance point-cloud map and the first and second camera road images, calibrates and fuses those to generate a road camera point-cloud fusion map, and then determines the road-line information of the second camera road image to generate a road-segmented map. The road-segmented map and the road camera point-cloud fusion map are fused. The distance data of the road-segmented map are obtained according to distance data points. A front driving path for the target road is planned according to the distance data.

Claims (39)

1. A path planning method comprising:

inputting a road distance point-cloud map, a first camera road image, and a second camera road image of a target road, wherein the road distance point-cloud map is created according to actual coordinates of a plurality of distance data points of the target road;

calibrating and fusing the road distance point-cloud map and the first camera road image to generate a road camera point-cloud fusion map and determining road-line information of the second camera road image to generate a road-segmented map of the target road; fusing the road-segmented map and the road camera point-cloud fusion map according to their feature points, thereby obtaining a plurality of distance data of the road-segmented map according to the plurality of distance data points, and planning a front driving path for the target road according to the plurality of distance data; and,

performing a path operation on the target road to plan the driving path when the number of the plurality of distance data points is larger than a threshold and a distance between each of the plurality of distance data points and the target road is less than a radius range.

2. The path planning method according to claim 1 , wherein the step of planning the driving path comprises:

computing the plurality of distance data points based on a vector quantization algorithm to determine initial road data of the target road;

clustering the initial road data based on a clustering algorithm; and

using circle equations and polynomials to fit a path of the target road, thereby planning the driving path.

3. The path planning method according to claim 2 , wherein the vector quantization algorithm is a k-means algorithm.

4. The path planning method according to claim 2 , wherein the clustering algorithm is density-based spatial clustering of applications with noise (DBSCAN).

5. The path planning method according to claim 1 , wherein the road distance point-cloud map is a road lidar point-cloud map.

6. The path planning method according to claim 5 , wherein the step of calibrating and fusing the road distance point-cloud map and the first camera road image comprises:

obtaining internal parameters of a camera for capturing the first camera road image and external parameters of the actual coordinates of the road distance point-cloud map and superposing the road distance point-cloud map and the first camera road image according to relationships between the internal parameters and the external parameters.

7. The path planning method according to claim 1 , wherein the step of determining the road-line information of the second camera road image to generate the road-segmented map comprises:

performing model initialization on the second camera road image to generate a model-initializing image;

segmenting the model-initializing image to generate a road-predicting map; and

computing the road-predicting map based on a probabilistic graphical model to generate a lane-segmented map and a road line-segmented map and fusing the lane-segmented map and the road line-segmented map to generate the road-segmented map.

8. A path planning system comprising:

an image-capturing device configured to capture a first camera road image and a second camera road image of a target road;

a point-cloud map-retrieving device configured to retrieve a plurality of distance data points of the target road and create a road distance point-cloud map according to actual coordinates of the plurality of distance data points; and

a processing device, connected to the image-capturing device and the point-cloud map-retrieving device, comprising:

a point-cloud image-fusing module configured to receive, calibrate, and fuse the road distance point-cloud map and the first camera road image to generate a road camera point-cloud fusion map;

a road area-recognizing module configured to receive the second camera road image and determine road-line information of the second camera road image to generate a road-segmented map of the target road; and

a path-planning module connected to the point-cloud image-fusing module and the road area-recognizing module and configured to receive the road-segmented map and fuse the road-segmented map and the road camera point-cloud fusion map according to their feature points, thereby obtaining a plurality of distance data of the road-segmented map according to the plurality of distance data points, and the path-planning module is configured to plan a front driving path for the target road according to the plurality of distance data,

wherein the path-planning module performs a path operation on the target road to plan the driving path when the number of the plurality of distance data points is larger than a threshold and a distance between each of the plurality of distance data points and the target road is less than a radius range.

9. The path planning system according to claim 8 , wherein the path-planning module is configured to:

compute the plurality of distance data points based on a vector quantization algorithm to determine initial road data of the target road;

cluster the initial road data based on a clustering algorithm; and

use circle equations and polynomials to fit a path of the target road, thereby planning the driving path.

10. The path planning system according to claim 9 , wherein the vector quantization algorithm is a k-means algorithm.

11. The path planning system according to claim 9 , wherein the clustering algorithm is density-based spatial clustering of applications with noise (DBSCAN).

12. The path planning system according to claim 8 , wherein the road distance point-cloud map is a road lidar point-cloud map.

13. The path planning system according to claim 8 , wherein the point-cloud image-fusing module is configured to obtain internal parameters of a camera for capturing the first camera road image and external parameters of the actual coordinates of the road distance point-cloud map and superpose the road distance point-cloud map and the first camera road image according to relationships between the internal parameters and the external parameters.

14. The path planning system according to claim 8 , wherein the road area-recognizing module is configured to:

perform model initialization on the second camera road image to generate a model-initializing image;

segment the model-initializing image to generate a road-predicting map; and

compute the road-predicting map based on a probabilistic graphical model to generate a lane-segmented map and a road line-segmented map and fusing the lane-segmented map and the road line-segmented map to generate the road-segmented map.

15. The path planning system according to claim 8 , wherein the image-capturing device is a camera.

16. The path planning system according to claim 8 , wherein the point-cloud map-retrieving device is an optical lidar.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2022
From: GUO, JIUN-IN; CHANG, JEN-SHUO
To: NATIONAL YANG MING CHIAO TUNG UNIVERSITY
Reel/Frame 058786/0220 →
Priority Claims (1)
TW 110142540 · Nov 16, 2021 · national
Continuity (1)
Related Publication 20230154203A1 · May 18, 2023
Cited By (1)
US 12,380,706