IP Library Granted Patent US 11,100,341
Granted Patent B2
US 11,100,341 · App. 16/509,112 · Granted Aug 24, 2021

Lane line tracking method and device

Inventors: Shufu Xie (Beijing, CN); Yuqiang Zhai (Beijing, CN); Tian Xia (Beijing, CN); Yu Ma (Beijing, CN)
Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.
G06K9/00798G06K9/44G08G1/167G06K9/00791G06T2207/30256
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,100,341
App. No.
16/509,112
Granted
Aug 24, 2021
Kind
B2
Abstract

A lane line tracking method and device are provided. The method can include: projecting a lane line detection result of previous K frames preceding a current frame in a camera coordinate system to a world coordinate system, to obtain a first projection result, wherein K is a positive integer greater than or equal to 1, and the previous K frames are consecutive K frames preceding the current frame; projecting the first projection result of the previous K frames in the world coordinate system to a camera coordinate system of the current frame, to obtain a second projection result; and determining, in the camera coordinate system of the current frame, lane line groups of the current frame, according to lane line groups in the second projection result of the previous K frames.

Claims (45)

1. A lane line tracking method, comprising:

projecting a lane line detection result of previous K frames preceding a current frame in a camera coordinate system to a world coordinate system, to obtain a first projection result, wherein K is a positive integer greater than or equal to 1, and the previous K frames are consecutive K frames preceding the current frame;

projecting the first projection result of the previous K frames in the world coordinate system to a camera coordinate system of the current frame, to obtain a second projection result; and

determining, in the camera coordinate system of the current frame, lane line groups of the current frame, according to lane line groups in the second projection result of the previous K frames.

2. The lane line tracking method of claim 1 , wherein the determining, in the camera coordinate system of the current frame, lane line groups of the current frame, according to lane line groups in the second projection result of the previous K frames comprises:

selecting a lane line group from the current frame, and calculating minimum distances between the selected lane line group of the current frame and respective lane line groups in the second projection result of the previous K frames; and

merging the selected lane line group of the current frame into an S-th lane line group in the second projection result of the previous K frames, if a minimum distance between the selected lane line group and the S-th lane line group in the second projection result of the previous K frames has a minimum of the minimum distances, wherein S is a positive integer greater than or equal to 1.

3. The lane line tracking method of claim 2 , wherein the merging the selected lane line group of the current frame into the S-th lane line group in the second projection result of the previous K frames comprises:

merging the selected lane line group of the current frame into the S-th lane line group in the second projection result of the previous K frames, if the minimum of the minimum distances is less than a preset threshold; and

establishing a new group based on the selected lane line group of the current frame, if the minimum of the minimum distances is greater than or equal to the preset threshold.

4. The lane line tracking method of claim 1 , further comprising:

performing, in the camera coordinate system of the current frame, a smoothing-process on lane line points of the lane line groups of the current frame, according to lane line points of the lane line groups in the second projection result of the previous K frames.

5. The lane line tracking method of claim 4 , wherein the performing a smoothing-process on lane line points of the lane line groups of the current frame, according to lane line points of the lane line groups in the second projection result of the previous K frames comprises:

filtering out a lane line point of a P-th lane line group of the current frame, if a distance between the lane line point of the P-th lane line group of the current frame and the P-th lane line group of the preceding frame are greater than a threshold, wherein P is a positive integer greater than or equal to 1.

6. The lane line tracking method of claim 1 , further comprising:

complementing the current frame by using lane line points of the lane line groups in the second projection result of the previous K frames.

7. The lane line tracking method of claim 6 , wherein the complementing the current frame by using lane line points of the lane line groups in the second projection result of the previous K frames comprises:

adding, in the current frame, lane line points of an M-th lane line group of the current frame by using lane line points of the M-th lane line group in the second projection result of the previous K frames, if the second projection result of the previous K frames comprises the M-th lane line group and the current frame does not comprise a group corresponding to the M-th lane line group, wherein M is a positive integer greater than or equal to 1.

8. The lane line tracking method of claim 1 , further comprising:

deleting an N-th lane line group in the second projection result of the previous K frames, if the second projection result of the previous K frames comprises the N-th lane line group and consecutive T frames succeeding the current frame does not comprise a group corresponding to the N-th lane line group, wherein T is a positive integer greater than 1, and N is a positive integer greater than or equal to 1.

9. A lane line tracking device, comprising:

one or more processors;

a storage device for storing one or more programs;

a camera for capturing images; wherein

the one or more programs are executed by the one or more processors to enable the one of more processors to implement a method comprising:

projecting a lane line detection result of previous K frames preceding a current frame in a camera coordinate system to a world coordinate system, to obtain a first projection result, wherein K is a positive integer greater than or equal to 1, and the previous K frames are consecutive K frames preceding the current frame;

projecting the first projection result of the previous K frames in the world coordinate system to a camera coordinate system of the current frame, to obtain a second projection result; and

determining, in the camera coordinate system of the current frame, lane line groups of the current frame, according to lane line groups in the second projection result of the previous K frames.

10. The lane line tracking device of claim 9 , wherein the one or more programs are executed by the one or more processors to enable the one or more processors to implement the method further comprising:

selecting a lane line group from the current frame, and calculate minimum distances between the selected lane line group of the current frame and respective lane line groups in the second projection result of the previous K frames; and

merging the selected lane line group of the current frame into an S-th lane line group in the second projection result of the previous K frames, if a minimum distance between the selected lane line group and the S-th lane line group in the second projection result of the previous K frames has a minimum of the minimum distances, wherein S is a positive integer greater than or equal to 1.

11. The lane line tracking device of claim 10 , wherein the one or more programs are executed by the one or more processors to enable the one or more process to implement the method further comprising:

merging the selected lane line group of the current frame into the S-th lane line group in the second projection result of the previous K frames, if the minimum of the minimum distances is less than a preset threshold; and

establishing a new group based on the selected lane line group of the current frame, if the minimum of the minimum distances is greater than or equal to the preset threshold.

12. The lane line tracking device of claim 10 , wherein the one or more programs are executed by the one or more processors to enable the one or more process to implement the method further comprising:

performing, in the camera coordinate system of the current frame, a smoothing-process on lane line points of the lane line groups of the current frame, according to lane line points of the lane line groups in the second projection result of the previous K frames.

13. The device of claim 12 , wherein the one or more programs are executed by the one or more processors to enable the one or more process to implement the method further comprising:

filtering out a lane line point of a P-th lane line group of the current frame, if a distance between the lane line point of the P-th lane line group of the current frame and the P-th lane line group of the preceding frame are greater than a threshold, wherein P is a positive integer greater than or equal to 1.

14. The lane line tracking device of claim 9 , wherein the one or more programs are executed by the one or more processors to enable the one or more process to implement the method further comprising:

complementing the current frame by using lane line points of the lane line groups in the second projection result of the previous K frames.

15. The lane line tracking device of claim 14 , wherein the one or more programs are executed by the one or more processors to enable the one or more process to implement the method further comprising:

adding, in the current frame, lane line points of an M-th lane line group of the current frame by using lane line points of the M-th lane line group in the second projection result of the previous K frames, if the second projection result of the previous K frames comprises the M-th lane line group and the current frame does not comprise a group corresponding to the M-th lane line group, wherein M is a positive integer greater than or equal to 1.

16. The lane line tracking device of claim 9 , wherein the one or more programs are executed by the one or more processors to enable the one or more process to implement the method further comprising:

deleting an N-th lane line group in the second projection result of the previous K frames, if the second projection result of the previous K frames comprises the N-th lane line group and consecutive T frames succeeding the current frame does not comprise a group corresponding to the N-th lane line group, wherein T is a positive integer greater than 1, and N is a positive integer greater than or equal to 1.

17. A non-transitory computer readable storage medium having computer programs stored thereon, wherein the programs, when executed by a processor, cause the processor to implement the method of claim 1 .

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICANT NAME PREVIOUSLY RECORDED AT REEL: 057933 FRAME: 0812. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058594/0836 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 057933/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2019
From: XIE, SHUFU; ZHAI, YUQIANG; XIA, TIAN; MA, YU
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 049837/0297 →
Priority Claims (1)
CN 20181102606.X · Sep 4, 2018 · national
Continuity (1)
Related Publication 20200074188A1 · Mar 5, 2020