IP Library Granted Patent US 10,317,231
Granted Patent B2
US 10,317,231 · App. 14/736,057 · Granted Jun 11, 2019

Top-down refinement in lane marking navigation

Inventors: Andras Ferencz (Munich, DE); Pini Reisman (Jerusalem, IL)
Assignee: Mobileye Vision Technologies Ltd.
G01C21/3602B60W30/12G05D1/0251G05D1/0253G06K9/00798G06K9/3241G06K9/6203G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,317,231
App. No.
14/736,057
Granted
Jun 11, 2019
Kind
B2
Abstract

Systems and methods use cameras to provide autonomous navigation features. In one implementation, top-down refinement in lane marking navigation is provided. The system may include one or more memories storing instructions and one or more processors configured to execute the instructions to cause the system to receive from one or more cameras one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line, update a model of the lane marking based on odometry of the one or more cameras relative to the roadway, refine the updated model of the lane marking based on an appearance of dashes derived from the received one or more images and a spacing between dashes derived from the received one or more images, and cause one or more navigational responses in the vehicle based on the refinement of the updated model.

Claims (55)

1. A computer system comprising:

one or more memories storing instructions; and

one or more processors configured to execute the instructions to cause the system to:

receive, from one or more cameras, one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line;

obtain a multi-frame model of the lane marking, wherein the multi-frame model of the lane marking includes a location for the lane marking at a first time;

update the multi-frame model of the lane marking based on odometry of the one or more cameras relative to the roadway to obtain an updated multi-frame model of the lane marking, wherein the updated multi-frame model of the lane marking includes an estimated updated location for the lane marking at a second time;

identify a plurality of candidate dashes from the received one or more images;

calculate spacing information relating to spaces between the plurality of candidate dashes;

perform a top-down modeling process to refine the updated multi-frame model of the lane marking at the updated location at the second time based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information; and

cause one or more navigational responses in the vehicle based on the refinement of the updated multi-frame model.

2. The computer system of claim 1 , wherein the refinement of the updated multi-frame model is further based on a lateral position, relative to the roadway, of dashes derived from the received one or more images.

3. The computer system of claim 1 , wherein the refinement of the updated multi-frame model is further based on existence of a parallel lane marking on one side of dashes derived from the received one or more images.

4. The computer system of claim 1 , wherein the appearance of the dashes is based on a width of the dashes.

5. The computer system of claim 1 , wherein the appearance of the dashes is based on a brightness of the dashes.

6. The computer system of claim 1 , wherein the appearance of the dashes is based on an edge sharpness of the dashes.

7. The computer system of claim 1 , wherein refining the updated multi-frame model of the lane marking based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information comprises:

refining the updated multi-frame model using a dynamic programming technique.

8. The computer system of claim 1 , wherein the one or more processors are configured to execute the instructions to cause the system to:

perform a bottom-up modeling process to generate a single-frame model of the lane marking based on a single frame of the received one or more images; and

match the generated single-frame model to the refined multi-frame model to approve the lane marking.

9. The computer system of claim 1 , wherein the one or more one navigational responses comprises a steering of the vehicle.

10. The computer system of claim 1 , wherein the one or more one navigational responses comprises an audible alert.

11. A vehicle, comprising:

a body;

one or more cameras configured to acquire one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line;

one or more memories storing instructions; and

one or more processors configured to execute the instructions to cause the vehicle to:

receive, from one or more cameras, the one or more acquired images of the roadway;

obtain a multi-frame model of the lane marking, wherein the multi-frame model of the lane marking includes a location for the lane marking at a first time;

update the multi-frame model of the lane marking based on odometry of the one or more cameras relative to the roadway to obtain an updated multi-frame model of the lane marking, wherein the updated multi-frame model of the lane marking includes an estimated updated location for the lane marking at a second time;

identify a plurality of candidate dashes from the received one or more images;

calculate spacing information relating to spaces between the plurality of candidate dashes;

perform a top-down modeling process to refine the updated multi-frame model of the lane marking at the updated location at the second time based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information; and

cause one or more navigational responses in the vehicle based on the refinement of the multi-frame updated model.

12. The vehicle of claim 11 , wherein the refinement of the multi-frame updated model is further based on a lateral position, relative to the roadway, of dashes derived from the received one or more images.

13. The vehicle of claim 11 , wherein the refinement of the multi-frame updated model is further based on existence of a parallel lane marking on one side of dashes derived from the received one or more images.

14. The vehicle of claim 11 , wherein the appearance of the dashes is based on a width of the dashes.

15. The vehicle of claim 11 , wherein the appearance of the dashes is based on a brightness of the dashes.

16. The vehicle of claim 11 , wherein the appearance of the dashes is based on an edge sharpness of the dashes.

17. The vehicle of claim 11 , wherein refining the updated multi-frame model of the lane marking based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information comprises:

refining the updated multi-frame model using a dynamic programming technique.

18. The vehicle of claim 11 , wherein the one or more processors are configured to execute the instructions to cause the vehicle to:

generate a model of the lane marking based on the received one or more images; and

match the generated model to the refined model to approve the lane marking.

19. The vehicle of claim 11 , wherein the one or more one navigational responses comprises a steering of the vehicle.

20. The vehicle of claim 11 , wherein the one or more one navigational responses comprises an audible alert.

21. A method comprising:

receiving, by one or more processors, from one or more cameras, one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line;

obtaining a multi-frame model of the lane marking, wherein the model of the lane marking includes a location for the lane marking at a first time;

updating, by the one or more processors, the multi-frame model of the lane marking based on odometry of the one or more cameras relative to the roadway to obtain an updated multi-frame model of the lane marking, wherein the updated multi-frame model of the lane marking includes an estimated updated location for the lane marking at a second time;

identifying a plurality of candidate dashes from the received one or more images;

calculating spacing information relating to spaces between the plurality of candidate dashes;

performing a top-down modeling process to refine, by the one or more processors, the updated multi-frame model of the lane marking at the updated location at the second time based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information; and

causing, by the one or more processors, one or more navigational responses in the vehicle based on the refinement of the updated multi-frame model.

22. The method of claim 21 , wherein the refinement of the updated multi-frame model is further based on a lateral position, relative to the roadway, of dashes derived from the received one or more images.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 14, 2020
From: FERENCZ, ANDRAS; REISMAN, PINI
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 052392/0810 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2015
From: FERENCZ, ANDRAS; REISMAN, PINI
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 036768/0928 →
Continuity (3)
Provisional Application 62173216 · Jun 9, 2015
Provisional Application 62010003 · Jun 10, 2014
Related Publication 20150354976A1 · Dec 10, 2015
Cited By (1)
US 12,384,410