IP Library › Granted Patent US 10,867,189
Granted Patent B2
US 10,867,189 · App. 15/659,384 · Granted Dec 15, 2020

Systems and methods for lane-marker detection

Inventors: Seungwoo Yoo (Yongin-si, KR); Duck Hoon Kim (Seoul, KR)
Assignee: QUALCOMM Incorporated
G06K9/00798G06T7/13G06T7/181G06T7/593G06K9/4604G06K9/4638G06T2207/10012G06T2207/30256H04N13/133H04N2013/0092
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,867,189
App. No.
15/659,384
Granted
Dec 15, 2020
Kind
B2
Abstract

An electronic device is described. The electronic device includes a memory and a processor in communication with the memory. The processor is configured to receive an image. The processor is also configured to extract a lane response map from the image. The processor is further configured to estimate one or more lane markers based on the lane response map.

Claims (58)

1. An electronic device, comprising:

a memory; and

a processor in communication with the memory, the processor configured to:

determine a red-green-blue (RGB)-to-grayscale conversion that maximizes intensity distances between a lane marker color and a road color in an image;

extract a lane response map from the RGB-to-grayscale conversion of the image the lane response map including a plurality of lane response values, wherein a lane response value for a pixel in the lane response map is determined based on a penalty component that reduces the lane response value as a difference between neighboring pixel values increases;

transform the lane response map to an inverse-perspective-mapping (IPM) domain;

define one or more sections in the IPM domain of the lane response map;

detect one or more straight lane-marker segments in the one or more sections based on a Radon transform of the lane response map in the IPM domain; and

cluster the one or more straight lane-marker segments to obtain one or more lane markers.

2. The electronic device of claim 1 , wherein defining the multiple sections is further based on at least one of a lane-marker segment width assumption, a pre-defined detection distance range, or calibration data.

3. The electronic device of claim 1 , wherein the processor is further configured to:

perform a transform in each of the multiple sections; and

obtain a lane-marker position and a lane-marker orientation based on the transform.

4. The electronic device of claim 1 , wherein the processor is further configured to filter one or more potential straight line segments to reduce false positives.

5. The electronic device of claim 1 , wherein the processor is further configured to refine the one or more lane markers.

6. The electronic device of claim 5 , wherein the processor is further configured to optimize a score function to obtain the one or more lane markers.

7. The electronic device of claim 1 , wherein the processor is further configured to apply a fitting to the one or more lane markers.

8. The electronic device of claim 1 , wherein the processor configured to extract the lane response map from the image comprises the processor configured to perform row-wise one-dimensional (1D) filtering on the image.

9. The electronic device of claim 1 , wherein the processor configured to determine the RGB-to-grayscale conversion that maximizes intensity distances between the lane marker color and the road color in the image comprises:

convert the image from a red-green-blue (RGB) color model to a grayscale color model; and

apply a trained projection model that maximizes intensity distances between the lane marker color and the road color using a linear discriminant analysis.

10. The electronic device of claim 1 , wherein the processor is further configured to apply an ego-lane mask to the image to restrict a lane-marker search region.

11. A method, comprising:

determining a red-green-blue (RGB)-to-grayscale conversion that maximizes intensity distances between a lane marker color and a road color in an image;

extracting a lane response map from the RGB-to-grayscale conversion of the image, the lane response map including a plurality of lane response values, wherein a lane response value for a pixel in the lane response map is determined based on a penalty component that reduces the lane response value as a difference between neighboring pixel values increases;

transforming the lane response map to an inverse-perspective-mapping (IPM) domain;

defining one or more sections in the IPM domain of the lane response map;

detecting one or more straight lane-marker segments in the one or more sections based on a Radon transform of the lane response map in the IPM domain; and

clustering the one or more straight lane-marker segments to obtain one or more lane markers.

12. The method of claim 11 , wherein extracting the lane response map from the image comprises performing row-wise one-dimensional (1D) filtering on the image.

13. The method of claim 11 , wherein determining the RGB-to-grayscale conversion that maximizes intensity distances between a lane marker color and a road color in the image comprises:

converting the image from a red-green-blue (RGB) color model to a grayscale color model; and

applying a trained projection model that maximizes intensity distances between the lane marker color and the road color using a linear discriminant analysis.

14. The method of claim 11 , further comprising applying an ego-lane mask to the image to restrict a lane-marker search region.

15. A non-transitory computer readable medium storing computer executable code, comprising:

code for causing an electronic device to determine a red-green-blue (RGB)-to-grayscale conversion that maximizes intensity distances between a lane marker color and a road color in an image;

code for causing the electronic device to extract a lane response map from the RGB-to-grayscale conversion of the image, the lane response map including a plurality of lane response values, wherein a lane response value for a pixel in the lane response map is determined based on a penalty component that reduces the lane response value as a difference between neighboring pixel values increases;

code for causing the electronic device to transform the lane response map to an inverse-perspective-mapping (IPM) domain;

code for causing the electronic device to define one or more sections in the IPM domain of the lane response map;

code for causing the electronic device to detect one or more straight lane-marker segments in the one or more sections based on a Radon transform of the lane response map in the IPM domain; and

code for causing the electronic device to cluster the one or more straight lane-marker segments to obtain one or more lane markers.

16. The computer readable medium of claim 15 , wherein the code for causing the electronic device to extract the lane response map from the image comprises performing row-wise one-dimensional (1D) filtering on the image.

17. The computer readable medium of claim 15 , wherein the code for causing the electronic device to determine the RGB-to-grayscale conversion that maximizes intensity distances between a lane marker color and a road color in the image comprises:

code for causing the electronic device to convert the image from a red-green-blue (RGB) color model to a grayscale color model; and

code for causing the electronic device to apply a trained projection model that maximizes intensity distances between the lane marker color and the road color using a linear discriminant analysis.

18. The computer readable medium of claim 15 , further comprising code for causing the electronic device to apply an ego-lane mask to the image to restrict a lane-marker search region.

19. An apparatus, comprising:

means for determining a red-green-blue (RGB)-to-grayscale conversion that maximizes intensity distances between a lane marker color and a road color in an image;

means for extracting a lane response map from the RGB-to-grayscale conversion of the image, the lane response map including a plurality of lane response values, wherein a lane response value for a pixel in the lane response map is determined based on a penalty component that reduces the lane response value as a difference between neighboring pixel values increases;

means for transforming the lane response map to an inverse-perspective-mapping (IPM) domain;

means for defining one or more sections in the IPM domain of the lane response map;

means for detecting one or more straight lane-marker segments in the one or more sections based on a Radon transform of the lane response map in the IPM domain; and

means for clustering the one or more straight lane-marker segments to obtain one or more lane markers.

20. The apparatus of claim 19 , wherein the means for extracting the lane response map from the image comprise means for performing row-wise one-dimensional (1D) filtering on the image.

21. The apparatus of claim 19 , wherein the means for determining the RGB-to-grayscale conversion that maximizes intensity distances between a lane marker color and a road color in the image comprise:

means for converting the image from a red-green-blue (RGB) color model to a grayscale color model; and

means for applying a trained projection model that maximizes intensity distances between the lane marker color and the road color using a linear discriminant analysis.

22. The apparatus of claim 19 , further comprising means for applying an ego-lane mask to the image to restrict a lane-marker search region.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2017
From: YOO, SEUNGWOO; KIM, DUCK HOON
To: QUALCOMM INCORPORATED
Reel/Frame 043346/0622 →
Continuity (2)
Provisional Application 62442420 · Jan 4, 2017
Related Publication 20180189577A1 · Jul 5, 2018