IP Library Granted Patent US 10,449,899
Granted Patent B2
US 10,449,899 · App. 15/147,301 · Granted Oct 22, 2019

Vehicle vision system with road line sensing algorithm and lane departure warning

Inventors: Nikhil Gupta (Brampton, CA); Eduardo R. Corral-Soto (Toronto, CA)
Assignee: MAGNA ELECTRONICS INC.
B60R1/00B60Q9/00G06K9/00798G06K9/209B60R2300/607B60R2300/804
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,449,899
App. No.
15/147,301
Granted
Oct 22, 2019
Kind
B2
Abstract

A vehicle vision system includes a plurality of cameras disposed at a vehicle and having respective fields of view exterior of the vehicle. An image processor is operable, responsive to image processing of captured image data, to synthesize captured image data to generate a reduced distortion bird's-eye view image of a region exterior and near the vehicle and encompassed by the fields of view of the cameras. A display displays the bird's-eye view image for viewing by a driver of the vehicle. The image processor processes synthesized captured image data using a lane marker sensing algorithm that utilizes recursive temporal stabilization to determine lane markings in the field of view of at least the rear camera. The system determines when the vehicle is departing from its being-traveled lane and, responsive to determination that the vehicle is departing from its being-traveled lane, generates an alert to the driver of the vehicle.

Claims (53)

1. A surround view vision system for a vehicle, said surround view vision system comprising:

a plurality of cameras disposed at a vehicle each having a respective field of view exterior of the vehicle that encompasses a respective ground region adjacent to the vehicle, said plurality of cameras comprising at least (i) a front camera disposed at a front portion of the vehicle and having at least a forward and downward field of view encompassing a ground region forward of the vehicle and (ii) a rear camera disposed at a rear portion of the vehicle and having at least a rearward and downward field of view encompassing a ground region rearward of the vehicle;

wherein each of said cameras comprises a lens and a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by at least some of said cameras;

wherein said image processor is operable, responsive to image processing of captured image data, to synthesize captured image data to generate a reduced distortion bird's-eye view image of a region exterior and near the vehicle and encompassed by the fields of view of said cameras;

a display that displays the bird's-eye view image for viewing by a driver of the vehicle while operating the vehicle;

wherein said image processor processes synthesized captured image data using a lane marker sensing algorithm to determine lane markings in the field of view of at least said rear camera;

wherein said image processor determines lane markers by (a) extracting low level image features from synthesized captured image data and (b) extracting line information from synthesized captured image data via pixel classification and line fitting;

wherein said lane marker sensing algorithm utilizes recursive temporal stabilization;

wherein said lane marker sensing algorithm predicts lane markers when they are no longer within the field of view of said rear camera;

wherein prediction of lane markers when they are no longer within the field of view of said rear camera is based at least in part on said recursive temporal stabilization; and

wherein said system, responsive to processing of synthesized captured image data using said lane marker sensing algorithm, determines when the vehicle is departing from its being-traveled lane and, responsive to a determination that the vehicle is departing from its being-traveled lane, generates an alert to the driver of the vehicle.

2. The surround view vision system of claim 1 , wherein said image processor processes synthesized captured image data and determines lane markings via classification and image processing techniques.

3. The surround view vision system of claim 1 , wherein a principal axis of the field of view of said rear camera is downward at an angle of about 45 degrees.

4. The surround view vision system of claim 1 , wherein said lens comprises a fish-eye lens.

5. The surround view vision system of claim 1 , wherein said image processor determines lane markers by stabilizing line information extracted from synthesized captured image data.

6. The surround view vision system of claim 1 , wherein said lane marker sensing algorithm utilizes said recursive temporal stabilization to reduce jitter of detected lines in synthesized captured image data.

7. The surround view vision system of claim 1 , wherein said lane marker sensing algorithm utilizes geometric road constraints to enhance determination of lane markers.

8. The surround view vision system of claim 1 , wherein said lane marker sensing algorithm utilizes synthesized captured image data to determine low-level features including image gradients and pixel-level statistics, and wherein said lane marker sensing algorithm utilizes a probabilistic pixel classification stage to classify data captured by each pixel of said rear camera as being part of a lane marker or not part of a lane marker.

9. The surround view vision system of claim 8 , wherein said lane marker sensing algorithm applies spatial line fitting to pixels that are determined to be part of a lane marker to determine lines in synthesized captured image data.

10. The surround view vision system of claim 9 , wherein said lane marker sensing algorithm applies road geometric constraints including lane marker width and gap size between lane markers.

11. The surround view vision system of claim 1 , wherein said alert comprises a visible alert generated in the displayed bird's-eye view image.

12. The surround view vision system of claim 11 , wherein said visible alert comprises a linear overlay that shows deviation of the vehicle's path of travel relative to the lane markers displayed in the displayed bird's-eye view image.

13. A surround view vision system for a vehicle, said surround view vision system comprising:

a plurality of cameras disposed at a vehicle each having a respective field of view exterior of the vehicle that encompasses a respective ground region adjacent to the vehicle, said plurality of cameras comprising at least (i) a rear camera disposed at a rear portion of the vehicle and having at least a rearward and downward field of view encompassing a ground region rearward of the vehicle, (ii) a driver side camera disposed at a driver-side portion of the vehicle and having at least a sideward and downward field of view encompassing a ground region sideward of the vehicle and (iii) a passenger side camera disposed at a passenger-side portion of the vehicle and having at least a sideward and downward field of view encompassing a ground region sideward of the vehicle;

wherein each of said cameras comprises a fish-eye lens and a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by at least some of said cameras;

wherein said image processor is operable, responsive to image processing of captured image data, to synthesize captured image data to generate a reduced distortion bird's-eye view image of a region exterior and near the vehicle and encompassed by the fields of view of said cameras;

a display that displays the bird's-eye view image for viewing by a driver of the vehicle while operating the vehicle;

wherein said image processor processes synthesized captured image data using a lane marker sensing algorithm to determine lane markings in the field of view of at least said rear camera;

wherein said image processor determines lane markers by (a) extracting low level image features from synthesized captured image data and (b) extracting line information from synthesized captured image data via pixel classification and line fitting;

wherein said lane marker sensing algorithm utilizes recursive temporal stabilization;

wherein said lane marker sensing algorithm predicts lane markers when they are no longer within the field of view of said rear camera;

wherein prediction of lane markers when they are no longer within the field of view of said rear camera is based at least in part on said recursive temporal stabilization;

wherein said system, responsive to processing of synthesized captured image data using said lane marker sensing algorithm, determines when the vehicle is departing from its being-traveled lane and, responsive to a determination that the vehicle is departing from its being-traveled lane, generates an alert to the driver of the vehicle; and

wherein said alert comprises a visible alert generated in the displayed bird's-eye view image, and wherein said visible alert comprises a linear overlay that shows deviation of the vehicle's path of travel relative to the lane markers displayed in the displayed bird's-eye view image.

14. The surround view vision system of claim 13 , wherein said lane marker sensing algorithm utilizes said recursive temporal stabilization to reduce jitter of detected lines in synthesized captured image data.

15. A surround view vision system for a vehicle, said surround view vision system comprising:

a plurality of cameras disposed at a vehicle each having a respective field of view exterior of the vehicle that encompasses a respective ground region adjacent to the vehicle, said plurality of cameras comprising at least (i) a rear camera disposed at a rear portion of the vehicle and having at least a rearward and downward field of view encompassing a ground region rearward of the vehicle, (ii) a driver side camera disposed at a driver-side portion of the vehicle and having at least a sideward and downward field of view encompassing a ground region sideward of the vehicle, (iii) a passenger side camera disposed at a passenger-side portion of the vehicle and having at least a sideward and downward field of view encompassing a ground region sideward of the vehicle, and (iv) a front camera disposed at a front portion of the vehicle and having at least a forward and downward field of view encompassing a ground region forward of the vehicle;

wherein each of said cameras comprises a fish-eye lens and a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by at least some of said cameras;

wherein said image processor is operable, responsive to image processing of captured image data, to synthesize captured image data to generate a reduced distortion bird's-eye view image of a region exterior and near the vehicle and encompassed by the fields of view of said cameras;

a display that displays the bird's-eye view image for viewing by a driver of the vehicle while operating the vehicle;

wherein said image processor processes synthesized captured image data using a lane marker sensing algorithm to determine lane markings in the field of view of at least said rear camera;

wherein said image processor determines lane markers by (a) extracting low level image features from synthesized captured image data and (b) extracting line information from synthesized captured image data via pixel classification and line fitting;

wherein said lane marker sensing algorithm utilizes recursive temporal stabilization;

wherein said lane marker sensing algorithm utilizes said recursive temporal stabilization to reduce jitter of detected lines in synthesized captured image data;

wherein said lane marker sensing algorithm predicts lane markers when they are no longer within the field of view of said rear camera;

wherein prediction of lane markers when they are no longer within the field of view of said rear camera is based at least in part on said recursive temporal stabilization;

wherein said system, responsive to processing of synthesized captured image data using said lane marker sensing algorithm, determines when the vehicle is departing from its being-traveled lane and, responsive to a determination that the vehicle is departing from its being-traveled lane, generates an alert to the driver of the vehicle;

wherein said alert comprises a visible alert generated in the displayed bird's-eye view image, and wherein said visible alert comprises a linear overlay that shows deviation of the vehicle's path of travel relative to the lane markers displayed in the displayed bird's-eye view image; and

wherein said alert further comprises an audible alert in the vehicle.

16. The surround view vision system of claim 15 , wherein said lane marker sensing algorithm utilizes geometric road constraints to enhance determination of lane markers, and wherein the utilized road geometric constraints include lane marker width and gap distance between markers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 10, 2016
From: GUPTA, NIKHIL; CORRAL-SOTO, EDUARDO R.
To: MAGNA ELECTRONICS INC.
Reel/Frame 038880/0561 →
Continuity (2)
Provisional Application 62158837 · May 8, 2015
Related Publication 20160325682A1 · Nov 10, 2016
Cited By (6)
US 12,286,154 US 12,311,930 US 12,319,281 US 12,337,894 US 12,400,353 US 12,670,720