IP Library Granted Patent US 10,753,758
Granted Patent B2
US 10,753,758 · App. 16/435,674 · Granted Aug 25, 2020

Top-down refinement in lane marking navigation

Inventors: Andras Ferencz (Princeton, NJ); Pinchas Reisman (Jerusalem, IL)
Assignee: MOBILEYE VISION TECHNOLOGIES LTD.
G01C21/3602B60W30/12G05D1/0251G05D1/0253G06K9/00798G06K9/3241G06K9/6203G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,753,758
App. No.
16/435,674
Granted
Aug 25, 2020
Kind
B2
Abstract

Systems and methods use cameras to provide autonomous navigation features. In one implementation, top-down refinement in lane marking navigation is provided. The system may include one or more memories storing instructions and one or more processors configured to execute the instructions to cause the system to receive from one or more cameras one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line, update a model of the lane marking based on odometry of the one or more cameras relative to the roadway, refine the updated model of the lane marking based on an appearance of dashes derived from the received one or more images and a spacing between dashes derived from the received one or more images, and cause one or more navigational responses in the vehicle based on the refinement of the updated model.

Claims (74)

1. A computer system comprising: one or more processors configured to execute instructions to cause the system to:

receive wirelessly, via a data interface, a plurality of images captured by an image acquisition device comprising at least three cameras, each of the least three cameras being configured to capture a distinct field of view, the plurality of images being of a roadway in a vicinity of a remote vehicle;

analyze the plurality of images to detect one or more segments of lane markings by;

deriving information from images captured by each of the at least three cameras,

determining whether derived information from different cameras of the at least three cameras is consistent, and

excluding the derived information from a selected camera, of the at least three cameras when determining the derived information from the selected camera is consistent with derived information from another camera;

generate a model based on the analysis of the plurality of images, the model representing the one or more detected segments

refine the model by performing a top-down modeling process based on:

an appearance of the one or more segments, the appearance comprising at least one of dimensions, texture, sharpness, or brightness of the one or more segments, and

a spacing calculated between segments derived from the plurality of images; and

transmit wirelessly the refined model to at least one vehicle for navigating the roadway based on refined model.

2. The system of claim 1 , wherein:

the data interface comprises a wireless link for receiving image data from the image acquisition device; and

the one or more processors and the vehicle are in different locations.

3. The system of claim 1 , wherein the plurality of images comprise a first set of images and a second set of images, the first set of images being received before the second set of images.

4. The system of claim 1 wherein to generate the model, the instructions cause the system to:

identify locations of the line markings at a first time; and

update the model based on odometry from the image acquisition device, the model comprising estimated updated locations of the line markings at a second time.

5. The system of claim 4 , wherein:

the model comprises a multi-frame model; and

the top-down modeling process comprises refining the multi-frame model based on the appearance of the one or more segments.

6. The system of claim 4 , wherein to generate the model, the instructions cause the system to:

identify a plurality of candidate dashes from the plurality of images; and

calculate spacing information between the plurality of candidate dashes.

7. The system of claim 1 , wherein the instructions further cause the system to:

generate navigational instructions based on the refined model; and

transmit wirelessly the navigational instructions to navigation systems positioned on a plurality of vehicles.

8. The system of claim 7 , wherein the navigation instructions comprise:

instructions to generate control signals for a throttling system;

instructions to generate control signals for a braking system; or

instructions to generate control signals for a steering system.

9. The system of claim 1 , wherein to analyze the plurality of images, the instructions cause the system to:

execute a stereo image analysis module; and

combine the plurality of images using the stereo image analysis module.

10. The system of claim 1 , wherein to analyze the plurality of images, the instructions cause the system to execute a monocular image analysis module.

11. The system of claim 1 , wherein to analyze the plurality of images, the instructions cause the system to:

detect features in the plurality of images, the features comprising at least one of vehicles, pedestrians, road signs, and highway exit ramps.

12. A computer implemented method comprising:

receiving wirelessly, via a data interface, a plurality of images captured by an image acquisition device wherein the image acquisition device comprises at least three cameras, each of the least three cameras being configured to capture a distinct field of view, the plurality of images being of a roadway in a vicinity of a remote vehicle;

analyzing the plurality of images to detect one or more segments of lane markings by:

deriving information from images captured by each of the at least three cameras,

determining whether derived information from different cameras of the at least three cameras is consistent, and

excluding the derived information from a selected camera of the at least three cameras when determining the derived information from the selected camera is consistent with derived information from another camera;

generating a model based on the analysis of the plurality of images, the model. representing the one or more detected segments, p‘refining the model by performing a top-down modeling process based on:

an appearance of the one or more segments, the appearance comprising at least one of dimensions, texture, sharpness, or brightness of the one or more segments, and

a spacing calculated between segments derived from the plurality of images; and

transmitting wirelessly the refined model to at least one vehicle for navigating the roadway based on the refined model.

13. The method of claim 12 , wherein:

the data interface comprises a wireless link for receiving image data from the image acquisition device; and

the one or more processors and the vehicle are in different locations.

14. The method of claim 12 , further comprising:

generating navigational instructions based on the refined model; and

transmitting wirelessly the navigational instructions to navigation systems positioned on a plurality of vehicles.

15. The method of claim 14 , wherein:

generating the model comprises:

updating the model based on odometry from the image acquisition device, the updated model comprising estimated updated locations of the line markings at a second time,

identifying a plurality of candidate dashed from the plurality of images, and

calculating spacing information between the plurality of candidate dashes;

the model comprises a multi-frame model; and

the top-down modeling process comprises refining the multi-frame model based on the appearance of the one or more segments.

16. The method of claim 12 , wherein analyzing the plurality of images comprises:

executing a stereo image analysis module; and

combining the plurality of images using the stereo image analysis module.

17. A non-transitory computer-readable medium storing instructions that, when executed by a processor, perform operations comprising:

receiving wirelessly, via a data interface, a plurality of images captured by an image acquisition device wherein the image acquisition device comprises at east three cameras, each of the east three cameras being configured to capture a distinct field of view, the plurality of images being of a roadway in a vicinity of a remote vehicle;

analyzing the plurality of images to detect one or more segments of lane markings by:

deriving information from images captured by each of the at least three cameras,

determining whether derived information from different cameras of the at least three cameras is consistent, and

excluding the derived information from a selected camera of the at least three cameras when determining the derived information from the selected camera is consistent with derived information from another camera;

generating a model based on the analysis of the plurality of images, the model representing the one or more detected segments;

refining the model by performing a top-down modeling process based on:

an appearance of the one or more segments, the appearance comprising at least one of dimensions, texture, sharpness, or brightness of the one or more segments, and

a spacing calculated between segments derived from the plurality of images; and

transmitting wirelessly the refined model to at least one vehicle for navigating the roadway based on the refined model.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 14, 2020
From: FERENCZ, ANDRAS; REISMAN, PINI
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 052392/0810 →
Continuity (4)
Continuation 14736057 · Jun 10, 2015
Provisional Application 62173216 · Jun 9, 2015
Provisional Application 62010003 · Jun 10, 2014
Related Publication 20200003573A1 · Jan 2, 2020
Cited By (1)
US 12,384,410