IP Library › Granted Patent US 11,436,744
Granted Patent B2
US 11,436,744 · App. 16/469,910 · Granted Sep 6, 2022

Method for estimating lane information, and electronic device

Inventors: Ji-man Kim (Suwon-si, KR); Chan-jong Park (Seoul, KR); Do-jun Yang (Yongin-si, KR); Hyun-woo Lee (Seoul, KR)
Assignee: Samsung Electronics Co., Ltd.
G06T7/536B60W50/00G06K9/6256G06T7/60G06V20/588G08G1/167B60W2050/0014B60W2420/42B60W2520/10B60W2552/00B60W2554/00B60W2554/4041B60W2555/60G06T2207/20081G06T2207/20084G06T2207/30236G06T2207/30256
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,436,744
App. No.
16/469,910
Granted
Sep 6, 2022
Kind
B2
Abstract

Provided is an Artificial Intelligence (AI) system for simulating a human brain's functions, such as recognition, decision, etc., by using a machine learning algorithm such as deep learning, etc. and applications of the AI system. Provided is an electronic device including: a camera configured to capture an outside image of a vehicle, and a processor configured to execute one or more instructions stored in a memory, wherein the processor executes the one or more instructions to: determine, from the captured image, at least one object for estimating lane information; estimate, from the image, lane information of a road on which the vehicle is traveling, based on a distance between the determined at least one object and the vehicle and a vanishing point of the image; and output guide information for guiding driving of the vehicle based on the estimated lane information.

Claims (68)

1. An electronic device comprising:

a camera configured to capture an outside image of a vehicle;

a memory storing one or more instructions; and

a processor configured to execute the one or more instructions stored in the memory,

wherein the processor executes the one or more instructions to:

determine, from the captured image, at least one object for estimating lane information, wherein the lane information comprises lines that divide a road on which the vehicle is driving along lanes,

determine a road area from the captured image,

acquire information about a number and determined different ratios of lanes of the road on which the vehicle is driving, based on information of the road at a location corresponding to position information of the vehicle,

estimate a lane width from the captured image, based on a distance between the determined at least one object and the vehicle, the determined road area, a vanishing point of the image, and the acquired information about the number of lanes,

divide the road area by the acquired number of lanes by using the vanishing point of the image as a pivot and the estimated lane width to estimate the lane information of the road, and

output guide information for guiding driving of the vehicle based on the estimated lane information,

wherein the lane information further comprises a first lane width of a first lane on which the vehicle is traveling and a lane position of the vehicle in the first lane,

wherein the guide information comprises a warning of a danger of lane departure,

wherein the information about the number and determined different ratios of lanes of the road on which the vehicle is driving is acquired based on at least one of global positioning system (GPS) position information of the vehicle, a local navigation included in the vehicle, or information stored in advance in the electronic device, and

wherein different respective widths of the lanes of the road are estimated based on the respective determined ratios of the lanes.

2. The electronic device of claim 1 , wherein the processor further executes the one or more instructions to:

acquire an artificial intelligence learning model based on a result obtained by using a plurality of images for a same subject to learn a relationship between the plurality of images for the same subject;

convert the captured image by using the acquired artificial intelligence learning model such that the captured image has visibility that is greater than or equal to a predetermined value; and

determine the at least one object for estimating the lane information from the converted image.

3. The electronic device of claim 1 , wherein the determined at least one object for estimating lane information comprises at least one among a guard rail, another vehicle driving ahead, or another vehicle driving behind.

4. The electronic device of claim 1 , wherein the processor further executes the one or more instructions to:

determine a driving center line of the vehicle;

estimate, when the determined at least one object includes a first front vehicle located on the driving center line, the first lane width of the first lane on which the vehicle is driving from the image, based on a distance between the first front vehicle and the vehicle and a vehicle width of the first front vehicle; and

estimate the lane information from the image based on the estimated first lane width, the vanishing point of the image, and the determined driving center line of the vehicle.

5. The electronic device of claim 4 , wherein the processor further executes the one or more instructions to:

estimate, when the determined at least one object includes a second front vehicle not located on the driving center line, a second lane width of a second lane on which the second front vehicle is driving, based on a distance between the second front vehicle and the driving center line, a distance between the second front vehicle and the vehicle, and a vehicle width of the second front vehicle; and

estimate the lane information based on the estimated second lane width, the vanishing point of the image, and a driving center line of the second front vehicle.

6. The electronic device of claim 1 , further comprising a display displaying the captured image,

wherein the processor further executes the one or more instructions to:

create the guide information based on the estimated lane information and a driving speed of the vehicle, and

synthesize the created guide information with the road area of the captured image to display a result of the synthesizing on the display.

7. The electronic device of claim 1 , wherein the processor further executes the one or more instructions to:

determine whether the position of the vehicle in the first lane is included in a predetermined range, based on the estimated lane information; and

output the guide information based on a result of the determining of the lane position of the vehicle in the first lane.

8. The electronic device of claim 1 , wherein the guide information is output in a form of displaying at least one of a text, an image, or an animation.

9. A method comprising:

acquiring an outside image of a vehicle;

determining, from the acquired image, at least one object for estimating lane information, wherein the lane information comprises lines that divide a road on which the vehicle is driving along lanes;

determining a road area from the acquired image;

acquiring information about a number and determined different ratios of lanes of the road on which the vehicle is driving, based on information of the road at a location corresponding to position information of the vehicle;

estimating a lane width from the acquired image, based on a distance between the determined at least one object and the vehicle, the determined road area, a vanishing point of the image, and the acquired information about the number of lanes;

dividing the road area by the acquired number of lanes by using the vanishing point of the image as a pivot and the estimated lane width to estimate the lane information of the road; and

outputting guide information for guiding driving of the vehicle based on the estimated lane information,

wherein the lane information further comprises a first lane width of a first lane on which the vehicle is traveling and a lane position of the vehicle in the first lane,

wherein the guide information comprises a warning of a danger of lane departure, and

wherein the information about the number and determined different ratios of lanes of the road on which the vehicle is driving is acquired based on at least one of global positioning system (GPS) position information of the vehicle, a local navigation included in the vehicle, or information stored in advance, and

wherein different respective widths of the lanes of the road are estimated based on the respective determined ratios of the lanes.

10. The method of claim 9 , further comprising:

learning a relationship among a plurality of images for a same subject by using the plurality of images for the same subject; and

converting the acquired image based on a result of the learning such that the acquired image has visibility that is greater than or equal to a predetermined value,

wherein the determining of the at least one object for estimating the lane information from the acquired image comprises:

determining the at least one object for estimating the lane information from the converted image.

11. The method of claim 9 , further comprising:

determining a driving center line of the vehicle,

wherein when the determined at least one object includes a first front vehicle located on the driving center line, and

wherein the estimating of the lane information of the road on which the vehicle is driving from the image based on the distance between the determined at least one object and the vehicle and the vanishing point of the image comprises:

estimating, from the image, the first lane width of the first lane on which the vehicle is driving, based on the distance between the first front vehicle and the vehicle and a vehicle width of the first front vehicle, and

estimating the lane information from the image, based on the estimated first lane width, the vanishing point of the image, and the determined driving center line of the vehicle.

12. The method of claim 11 , wherein when the determined at least one object includes a second front vehicle not located on the driving center line, the estimating of the lane information from the image, based on the distance between the determined at least one object and the vehicle and the vanishing point of the image comprises:

estimating, from the image, a second lane width of a second lane on which the second front vehicle is driving, based on a distance between the second front vehicle and the driving center line, a distance between the second front vehicle and the vehicle, and a vehicle width of the second front vehicle; and

estimating the lane information from the image based on the estimated second lane width, the vanishing point of the image, and a driving center line of the second front vehicle.

13. The method of claim 9 , wherein the outputting of the guide information for guiding driving of the vehicle, based on the estimated lane information, comprises:

creating the guide information based on the estimated lane information and a driving speed of the vehicle; and

synthesizing the created guide information with the road area of the acquired image to display a result of the synthesizing.

14. The method of claim 9 , wherein the outputting of the guide information for guiding driving of the vehicle, based on the estimated lane information, comprises:

determining whether the position of the vehicle in the first lane is included in a predetermined range, based on the estimated lane information; and

outputting the guide information based on a result of the determining of the lane position of the vehicle in the first lane.

15. A computer-readable recording medium storing a program for executing the method of claim 9 , on a computer.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2019
From: KIM, JI-MAN; PARK, CHAN-JONG; YANG, DO-JUN; LEE, HYUN-WOO
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 049472/0660 →
Priority Claims (2)
KR 10-2016-0178012 · Dec 23, 2016 · national
KR 10-2017-0142567 · Oct 30, 2017 · national
Continuity (1)
Related Publication 20190318492A1 · Oct 17, 2019
Cited By (1)
US 12,412,390