Method, apparatus, electronic device, computer program and computer-readable recording medium for detecting lane marking based on vehicle image
There is provided a method for detecting a lane marking using a processor including acquiring a drive image captured by an image capturing device of a vehicle which is running, detecting an edge corresponding to a lane marking from the acquired drive image and generating an edge image based on the detected edge, detecting a linear component based on the detected edge and generating a linearly processed edge image based on the detected linear component, detecting a lane marking point corresponding to the lane marking using the generated edge image and the linearly processed edge image, and detecting the lane marking based on the detected lane marking point.
1 . An electronic device located in a vehicle, comprising:
a camera;
a display;
a head up display (HUD) for displaying a plurality of visual objects on a windshield of the vehicle;
memory comprising one or more storage media, storing instructions; and
at least one processor comprising processing circuitry,
wherein the instructions, when executed by the at least one processor, cause the electronic device to:
while displaying a first user interface on the display, identify an input on an executable icon configured to execute an augmented reality (AR) view mode, included in the first user interface,
based on identifying the input on the executable icon,
obtain, using the camera, a first image of a front view of the vehicle,
generate, based on the first image, a second image of a top view of the vehicle,
identify, based on the second image, a first element corresponding to a left lane marking and a second element corresponding to a right lane marking,
identify, based on the first element and the second element, within the second image, a first point corresponding to an initial position of the first element, and a second point corresponding to an initial position of the second element,
determine, based on the first point and the second point, first candidate areas and second candidate areas, wherein each first candidate area of the first candidate areas includes a plurality of blocks, and wherein each second candidate area of the second candidate areas includes a plurality of blocks,
identify a plurality of third points related to the left lane marking within the first candidate areas by determining a block among a plurality of blocks included in each first candidate area, and identify a plurality of fourth points related to the right lane marking within the second candidate areas by determining a block among a plurality of blocks included in each second candidate area,
identify, based on the plurality of third points and the plurality of fourth points, a plurality of fifth points, wherein each of the plurality of fifth points is a center point of one of the plurality of third points and one of the plurality of fourth in the second image, and
display, via the HUD, on the windshield, a second user interface including a first visual object along a first line identified based on the plurality of third points, a second visual object along a second line identified based on the plurality of fourth points, and a third visual object for guiding a direction to a destination along a third line identified based on the plurality of fifth points, in the AR view mode,
wherein the first visual object is overlaid on the windshield and is superimposed on the left lane marking visible through the windshield,
wherein the second visual object is overlaid on the windshield and is superimposed on the right lane marking visible through the windshield, and
wherein the third visual object is overlaid centrally between the left lane marking and the right lane marking visible through the windshield.
2 . The electronic device of claim 1 , wherein the first visual object is displayed with a designated width along the first line corresponding to the left lane marking in a driving area formed between the first line and the second line, and
wherein the second visual object is displayed with the designated width along the second line corresponding to the right lane marking in the driving area.
3 . The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor, cause the electronic device to:
identify, based on the first line and the second line, that the vehicle is deviated from one of the left lane marking and the right lane marking, and
in response to identifying that the vehicle is deviated from one of the left lane marking and the right lane marking, cease displaying the first visual object and the second visual object and display a fourth visual object superimposed on the one of the left lane marking and the right lane marking.
4 . The electronic device of claim 3 , wherein the instructions, when executed by the at least one processor, cause the electronic device to maintain displaying the third visual object based on identifying that the vehicle is deviated from one of the left lane marking and the right lane marking.
5 . The electronic device of claim 4 , wherein the instructions, when executed by the at least one processor, cause the electronic device to provide driving warning while the vehicle is operated in an autonomous driving mode of the driving mode and a manual mode, in response to identifying that the vehicle is deviated from one of the left lane marking and the right lane marking.
6 . The electronic device of claim 5 , wherein the instructions, when executed by the at least one processor, cause the electronic device to:
display the third visual object being configured for guiding a lane change,
based on displaying the third visual object being configured for guiding the lane change, identify that the vehicle is deviated from one of the left lane marking and the right lane marking as a normal driving state, and
based on identifying that the vehicle is deviated from one of the left lane marking and the right lane marking as the normal driving state, maintain displaying the third visual object.
7 . The electronic device of claim 6 , wherein the instructions, when executed by the at least one processor, cause the electronic device to display the first visual object and the second visual object after the vehicle returns to driving area formed between the first line and the second line.
8 . The electronic device of claim 1 , wherein the third visual object is displayed as an arrow shape.
9 . The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor, cause the electronic device to:
identify a plurality of areas on the display on which the first visual object, the second visual object, and the third visual object are displayed,
display driving information of the vehicle on a first area of the plurality of areas, and
display a map including an element to display a location of the vehicle on a second area of the plurality of areas.
10 . A method of an electronic device located in a vehicle, comprising:
while displaying a first user interface on a display of the electronic device, identify an input on an executable icon configured to execute an augmented reality (AR) view mode, included in the first user interface,
based on identifying the input on the executable icon, obtaining, using a camera of the electronic device, a first image of a front view of the vehicle,
generating, based on the first image, a second image of a top view of the vehicle,
identifying, based on the second image, a first element corresponding to a left lane marking and a second element corresponding to a right lane marking,
identifying, based on the first element and the second element, within the second image, a first point corresponding to an initial position of the first element, and a second point corresponding to an initial position of the second element,
determining, based on the first point and the second point, first candidate areas and second candidate areas, wherein each first candidate area of the first candidate areas includes a plurality of blocks, and wherein each second candidate area of the second candidate areas includes a plurality of blocks,
identifying a plurality of third points related to the left lane marking within the first candidate areas by determining a block among a plurality of blocks included in each first candidate area, and identify a plurality of fourth points related to the right lane marking within the second candidate areas by determining a block among a plurality of blocks included in each second candidate area,
identifying, based on the plurality of third points and the plurality of fourth points, a plurality of fifth points, wherein each of the plurality of fifth points is a center point of one of the plurality of third points and one of the plurality of fourth points in the second image, and
displaying, via a head up display (HUD) of the electronic device for displaying a plurality of visual objects on a windshield of the vehicle, a second user interface on the windshield in the AR view mode, the second user interface including, a first visual object along a first line identified based on the plurality of third points, a second visual object along a second line identified based on the plurality of fourth points, and a third visual object for guiding a direction to a destination along a third line identified based on the plurality of fifth points,
wherein the first visual object is overlaid on the windshield and is superimposed on the left lane marking visible through the windshield,
wherein the second visual object is overlaid on the windshield and is superimposed on the right lane marking visible through the windshield, and
wherein the third visual object is overlaid centrally between the left lane marking and the right lane marking visible through the windshield.
11 . The method of claim 10 , wherein the first visual object is displayed with a designated width along the first line corresponding to the left lane marking in a driving area formed between the first line and the second line, and
wherein the second visual object is displayed with the designated width along the second line corresponding to the right lane marking in the driving area.
12 . The method of claim 10 , the method further comprises:
identifying, based on the first line and the second line, that the vehicle is deviated from one of the left lane marking and the right lane marking, and
in response to identifying that the vehicle is deviated from one of the left lane marking and the right lane marking, ceasing displaying the first visual object and the second visual object and display a fourth visual object superimposed on the one of the left lane marking and the right lane marking.
13 . The method of claim 12 , the method further comprises maintaining displaying the third visual object based on identifying that the vehicle is deviated from one of the left lane marking and the right lane marking.
14 . The method of claim 13 , the method further comprises providing driving warning while the vehicle is operated in an autonomous driving mode of the driving mode and a manual mode, in response to identifying that the vehicle is deviated from one of the left lane marking and the right lane marking.
15 . The method of claim 13 , the method further comprises:
displaying the third visual object being configured for guiding a lane change,
based on displaying the third visual object being configured for guiding the lane change, identifying that the vehicle is deviated from one of the left lane marking and the right lane marking as a normal driving state, and
based on identifying that the vehicle is deviated from one of the left lane marking and the right lane marking as the normal driving state, maintaining displaying the third visual object.
16 . The method of claim 15 , the method further comprises displaying the first visual object and the second visual object after the vehicle returns to driving area formed between the first line and the second line.