IP Library Granted Patent US 11,727,693
Granted Patent B2
US 11,727,693 · App. 17/222,212 · Granted Aug 15, 2023

Method, apparatus, electronic device, computer program and computer-readable recording medium for detecting lane marking based on vehicle image

Inventor: Shin Hyoung Kim (Seongnam-si, KR)
Assignee: THINKWARE CORPORATION
G06V20/588G06V10/44G06V10/755
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,727,693
App. No.
17/222,212
Granted
Aug 15, 2023
Kind
B2
Abstract

There is provided a method for detecting a lane marking using a processor including acquiring a drive image captured by an image capturing device of a vehicle which is running, detecting an edge corresponding to a lane marking from the acquired drive image and generating an edge image based on the detected edge, detecting a linear component based on the detected edge and generating a linearly processed edge image based on the detected linear component, detecting a lane marking point corresponding to the lane marking using the generated edge image and the linearly processed edge image, and detecting the lane marking based on the detected lane marking point.

Claims (75)

1. A method for detecting a lane marking using a processor, the method comprising:

acquiring a drive image captured by an image capturing device of a vehicle which is running;

detecting an edge corresponding to a lane marking from the acquired drive image and generating an edge image based on the detected edge;

detecting a linear component based on the detected edge and generating a linearly processed edge image based on the detected linear component;

detecting a lane marking point corresponding to the lane marking using the generated edge image and the linearly processed edge image; and

detecting the lane marking based on the detected lane marking point,

wherein the detecting of a lane marking point includes:

detecting an initial lane marking point using at least one of the generated edge image and the linearly processed edge image; and

generating a search window, applying the generated search window to each of the generated edge image and the linearly processed edge image, and detecting a subsequent lane marking point satisfying a preset condition from the applied search window.

2. The method of claim 1 , further comprising:

generating a top view image by transforming a view of a front view image corresponding to the acquired drive image,

wherein, in the detecting of an edge, an edge corresponding to the lane marking may be detected from the top view image.

3. The method of claim 2 , further comprising:

dividing the top view image into a first region and a second region according to a distance from the vehicle,

wherein, in the generating of a linearly processed edge image, the linear component is detected from each of a first edge image corresponding to the divided first region and a second edge image corresponding to the second region, and a linearly processed edge image is generated based on the detected linear component.

4. The method of claim 1 , wherein,

in the detecting of a lane marking point, a lane marking point corresponding to a left lane marking and a lane marking point corresponding to a right lane marking are separately detected.

5. The method of claim 1 , wherein

the determining of an initial lane marking point includes:

detecting a left initial lane marking point at which a brightness value of a left side is maximized at a lower end of the edge image; and

detecting a right initial lane marking point at which a brightness value of a right side is maximized at the lower end of the edge image.

6. The method of claim 1 , wherein

the detecting of a subsequent lane marking point includes:

generating a first search window based on the determined initial lane marking point;

detecting a first subsequent lane marking point satisfying a preset condition by applying the generated first search window to the edge image and the linearly processed edge image;

generating a second search window in which a row in which the first subsequent lane marking point is located in the first search window, is set to a central row; and

disposing the generated second search window above the first search window and detecting a second subsequent lane marking point satisfying the preset condition by applying the second search window to each of the edge image and the linearly processed edge image.

7. The method of claim 1 , wherein

a lane marking point satisfying the preset condition is detected only from a middle row of the search window.

8. The method of claim 1 , wherein

a row size of the search window is changed adaptively according to an inclination of the linear component.

9. The method of claim 1 , further comprising:

calculating an energy value of an active contour model (ACM), wherein the preset condition is a lane marking point at which the energy value of the ACM is minimized.

10. The method of claim 9 , wherein

the calculating of an ACM energy value includes:

calculating an internal energy value using an average distance between previous lane marking points and a distance between a current lane marking point and a previous lane marking point; and

calculating an external energy value using brightness values of lane marking points detected from each of the edge image and the linearly processed edge image.

11. The method of claim 1 , wherein

the detecting of a lane marking further includes:

calculating a curve equation formed by the detected lane marking points; and

approximating the lane marking points to a curve based on the calculated curve equation.

12. The method of claim 2 , wherein

the detecting of a lane marking includes:

detecting lane marking points of the drive image corresponding to lane marking points of the top view image by transforming the lane marking points detected from the top view image into a view of the acquired drive image; and

detecting a lane marking based on the detected lane marking points of the drive image.

13. A lane marking detecting apparatus, the apparatus comprising a processor,

the processor comprising:

an image acquiring unit acquiring a drive image captured by an image capturing device of a vehicle which is running;

an edge detecting unit detecting an edge corresponding to a lane marking from the acquired drive image and generating an edge image based on the detected edge;

a linear component detecting unit detecting a linear component based on the detected edge and generating a linearly processed edge image based on the detected linear component;

a lane marking point detecting unit detecting a lane marking point corresponding to the lane marking using the generated edge image and the linearly processed edge image; and

a lane marking detecting unit detecting the lane marking based on the detected lane marking point,

wherein the lane marking point detecting unit includes:

an initial lane marking point detecting unit detecting an initial lane marking point using at least one of the generated edge image and the linearly processed edge image; and

a subsequent lane marking point detecting unit generating a search window, applying the generated search window to each of the generated edge image and the linearly processed edge image, and detecting a subsequent lane marking point satisfying a preset condition from the applied search window.

14. The lane marking detecting apparatus of claim 13 , the processor further comprising:

an image view transforming unit generating a top view image by transforming a view of a front view image corresponding to the acquired drive image,

wherein the edge detecting unit detects an edge corresponding to the lane marking from the top view image.

15. The lane marking detecting apparatus of claim 14 , the processor further comprising:

an image region dividing unit dividing the top view image into a first region and a second region according to a distance from the vehicle,

wherein the linear component is detected from each of a first edge image corresponding to the divided first region and a second edge image corresponding to the second region, and a linearly processed edge image is generated based on the detected linear component.

16. The lane marking detecting apparatus of claim 13 , the processor further comprising:

an active contour model energy (ACM) calculating unit calculating an energy value of an ACM,

wherein the preset condition is a lane marking point at which the energy value of the ACM is minimized.

17. The lane marking detecting apparatus of claim 13 , the processor further comprising:

a lane marking point approximating unit calculating a curve equation formed by the detected lane marking points and approximating the lane marking points to a curve based on the calculated curve equation.

18. A non-transitory computer readable storage medium containing instructions, that when executed by one or more processors, cause the one or more processor to perform a lane marking detecting method, the method comprising:

acquiring a drive image captured by an image capturing device of a vehicle which is running;

detecting an edge corresponding to a lane marking from the acquired drive image and generating an edge image based on the detected edge;

detecting a linear component based on the detected edge and generating a linearly processed edge image based on the detected linear component;

detecting a lane marking point corresponding to the lane marking using the generated edge image and the linearly processed edge image; and

detecting the lane marking based on the detected lane marking point,

wherein the detecting of a lane marking point includes:

detecting an initial lane marking point using at least one of the generated edge image and the linearly processed edge image; and

generating a search window, applying the generated search window to each of the generated edge image and the linearly processed edge image, and detecting a subsequent lane marking point satisfying a preset condition from the applied search window.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 5, 2021
From: KIM, SHIN HYOUNG
To: THINKWARE CORPORATION
Reel/Frame 055823/0915 →
Priority Claims (1)
KR 10-2020-0044088 · Apr 10, 2020 · national
Continuity (1)
Related Publication 20210319237A1 · Oct 14, 2021
Cited By (5)
US 1,056,938 US 1,056,939 US 1,056,941 US 1,058,586 US 1,123,950