IP Library › Granted Patent US 11,155,210
Granted Patent B2
US 11,155,210 · App. 16/949,048 · Granted Oct 26, 2021

Vehicular driving assist system including multiple cameras and radar sensor

Inventors: Lee Karner (Holland, MI); Kenneth Schofield (Holland, MI); Niall R. Lynam (Holland, MI)
Assignee: MAGNA ELECTRONICS INC.
B60R1/00B29C45/0017B29C45/14754B60C9/00B60Q9/00B60Q9/008B60R1/04B60R1/06B60R1/12B60R11/04G06K9/00805G06T7/13G06T7/70G08G1/165G08G1/166G08G1/167G09G3/001G09G3/346G09G5/026H04N7/181H04N7/183B29C45/14639B29C45/1671B29L2011/0058B29L2031/30B60R2001/1215B60R2001/1223B60R2001/1253B60R2001/1269B60R2011/0033B60R2300/105B60R2300/308B60R2300/808B60R2300/8026B60R2300/8066B60R2300/8093G06T2207/10016G06T2207/30232G06T2207/30261G09G2300/023G09G2300/0456G09G2340/12G09G2380/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,155,210
App. No.
16/949,048
Granted
Oct 26, 2021
Kind
B2
Abstract

A vehicular driving assist system includes a rearward viewing camera, a driver-side camera, a passenger-side camera and at least one radar sensor. Image data captured by the cameras is provided to a control and radar data captured by the radar sensor is provided to the control. The control processes the provided captured data to determine a potential hazard presented by rear-approaching traffic. The control outputs video image data for display by a display device of the equipped vehicle. The display device includes a video display screen that displays video images for viewing by a driver of the equipped vehicle during operation of the equipped vehicle. The control processes provided captured image data and provided captured radar data to alert the driver to the determined potential hazard presented by the rear-approaching traffic in the side traffic lane adjacent to the traffic lane being traveled by the equipped vehicle.

Claims (91)

1. A vehicular driving assist system, said vehicular driving assist system comprising:

a rearward viewing camera disposed at a rear portion of a vehicle equipped with said vehicular driving assist system;

said rearward viewing camera having a field of view at least rearward of the equipped vehicle;

said rearward viewing camera capturing frames of image data during operation of the equipped vehicle;

a driver-side camera disposed at a driver side portion of the equipped vehicle and a passenger-side camera disposed at a passenger side portion of the equipped vehicle;

wherein said driver-side camera has at least a sideward field of view external of the equipped vehicle, and wherein said passenger-side camera has at least a sideward field of view external of the equipped vehicle;

wherein said driver-side camera captures frames of image data during operation of the equipped vehicle;

wherein said passenger-side camera captures frames of image data during operation of the equipped vehicle;

at least one radar sensor disposed at the equipped vehicle;

the at least one radar sensor having a field of sensing external of the equipped vehicle;

wherein the at least one radar sensor captures radar data during operation of the equipped vehicle;

a control disposed in the equipped vehicle, wherein said control comprises a data processor;

wherein image data captured by said rearward viewing camera is provided to said control, and wherein image data captured by said driver-side camera is provided to said control, and wherein image data captured by said passenger-side camera is provided to said control, and wherein radar data captured by the at least one radar sensor is provided to said control;

wherein captured image data provided to said control is processed at said control, and wherein captured radar data provided to said control is processed at said control;

wherein processing at said control of provided captured image data comprises use of an edge detection algorithm;

wherein, via processing at said control of provided captured image data and provided captured radar data, a potential hazard presented by rear-approaching traffic approaching the equipped vehicle in a traffic lane adjacent to a traffic lane being traveled by the equipped vehicle is determined;

wherein said control outputs video image data for display by a display device of the equipped vehicle, said display device comprising a video display screen that displays video images for viewing by a driver of the equipped vehicle during operation of the equipped vehicle;

wherein said display device is operable to display video images at said video display screen with a display intensity, as viewed by the driver of the equipped vehicle, greater than 200 candelas/sq. meter; and

wherein said control processes provided captured image data and provided captured radar data to alert the driver to the determined potential hazard presented by the rear-approaching traffic in the traffic lane adjacent to the traffic lane being traveled by the equipped vehicle.

2. The vehicular driving assist system of claim 1 , wherein said video display screen is disposed at an interior rearview mirror assembly of the equipped vehicle.

3. The vehicular driving assist system of claim 1 , wherein said control connects to a vehicle network of the equipped vehicle.

4. The vehicular driving assist system of claim 3 , wherein said rearward viewing camera comprises a CMOS imaging array.

5. The vehicular driving assist system of claim 4 , wherein said driver-side camera comprises a CMOS imaging array, and wherein said passenger-side camera comprises a CMOS imaging array.

6. The vehicular driving assist system of claim 1 , wherein, via processing at said control of provided captured image data and provided captured radar data, presence of an obstacle exterior the equipped vehicle is determined.

7. The vehicular driving assist system of claim 1 , wherein, via processing at said control of provided captured image data and provided captured radar data, presence of a person exterior the equipped vehicle is determined.

8. The vehicular driving assist system of claim 1 , wherein data processing at said control comprises fusion of provided captured image data and provided captured radar data.

9. The vehicular driving assist system of claim 1 , wherein said rearward viewing camera comprises a rear backup camera, and wherein during a reversing maneuver of the equipped vehicle, said rear backup camera views at least a portion of a vehicle bumper of the equipped vehicle.

10. The vehicular driving assist system of claim 9 , wherein the equipped vehicle comprises a trailer hitch disposed at a rear portion of the equipped vehicle, and wherein, when the driver of the equipped vehicle is maneuvering the equipped vehicle to hitch a trailer to the equipped vehicle, said control processes image data captured by said rear backup camera and outputs for display by said video display screen video images derived from image data captured by said rear backup camera.

11. The vehicular driving assist system of claim 10 , wherein, responsive to said control processing image data captured by said rear backup camera, at least one overlay overlays the displayed video images to guide the driver when hitching the trailer to the trailer hitch of the equipped vehicle.

12. The vehicular driving assist system of claim 11 , wherein the at least one overlay aids in guiding connection of the trailer hitch of the equipped vehicle to a trailer tongue of the trailer.

13. The vehicular driving assist system of claim 1 , wherein said video display screen of said display device comprises a backlit thin film transistor LCD video display screen backlit by at least one white light emitting light emitting diode.

14. The vehicular driving assist system of claim 1 , wherein said video display screen comprises an organic light emitting diode (OLED) video display screen.

15. The vehicular driving assist system of claim 1 , wherein said display device is operable to display video images at said video display screen with a display intensity, as viewed by the driver of the equipped vehicle, greater than 400 candelas/sq. meter.

16. The vehicular driving assist system of claim 1 , wherein said display device is disposed at an interior rearview mirror assembly of the equipped vehicle, and wherein at least one of (a) said display device displays video images for viewing through a mirror reflective element of said interior rearview mirror assembly by the driver of the equipped vehicle and (b) said display device automatically extends from said interior rearview mirror assembly responsive to the vehicle transmission being shifted into reverse gear, and wherein said interior rearview mirror assembly comprises a molded plastic mounting arm having at least one ball member at at least one end thereof and having electrical conducting elements extending therethrough, and wherein a mirror mounting portion is molded over said ball member, and wherein said mounting arm comprises a mounting arm polymeric material having a first linear mold shrinkage factor and a first flexural modulus, and wherein said mirror mounting portion comprises a mirror mounting polymeric material having a second linear mold shrinkage factor and a second flexural modulus, and wherein said second linear mold shrinkage factor is greater than said first linear mold shrinkage factor, and wherein said second flexural modulus is lower than said first flexural modulus.

17. A vehicular driving assist system, said vehicular driving assist system comprising:

a rearward viewing camera disposed at a rear portion of a vehicle equipped with said vehicular driving assist system;

said rearward viewing camera having a field of view at least rearward of the equipped vehicle;

wherein said rearward viewing camera comprises a CMOS imaging array;

said rearward viewing camera capturing frames of image data during operation of the equipped vehicle;

a driver-side camera disposed at a driver side portion of the equipped vehicle and a passenger-side camera disposed at a passenger side portion of the equipped vehicle;

wherein said driver-side camera has at least a sideward field of view external of the equipped vehicle, and wherein said passenger-side camera has at least a sideward field of view external of the equipped vehicle;

wherein said driver-side camera comprises a CMOS imaging array;

wherein said passenger-side camera comprises a CMOS imaging array;

wherein said driver-side camera captures frames of image data during operation of the equipped vehicle;

wherein said passenger-side camera captures frames of image data during operation of the equipped vehicle;

at least one radar sensor disposed at the equipped vehicle;

the at least one radar sensor having a field of sensing external of the equipped vehicle;

wherein the at least one radar sensor captures radar data during operation of the equipped vehicle;

a control disposed in the equipped vehicle, wherein said control comprises a data processor;

wherein image data captured by said rearward viewing camera is provided to said control, and wherein image data captured by said driver-side camera is provided to said control, and wherein image data captured by said passenger-side camera is provided to said control, and wherein radar data captured by the at least one radar sensor is provided to said control;

wherein captured image data provided to said control is processed at said control, and wherein captured radar data provided to said control is processed at said control;

wherein processing at said control of provided captured image data comprises use of an edge detection algorithm;

wherein, via processing at said control of provided captured image data and provided captured radar data, presence of a person exterior the equipped vehicle is determined;

wherein said control connects to a vehicle network of the equipped vehicle;

wherein, via processing at said control of provided captured image data and provided captured radar data, a potential hazard presented by rear-approaching traffic approaching the equipped vehicle in a traffic lane adjacent to a traffic lane being traveled by the equipped vehicle is determined;

wherein said control outputs video image data for display by a display device of the equipped vehicle, said display device comprising a video display screen that displays video images for viewing by a driver of the equipped vehicle during operation of the equipped vehicle; and

wherein said control processes provided captured image data and provided captured radar data to alert the driver to the determined potential hazard presented by the rear-approaching traffic in the traffic lane adjacent to the traffic lane being traveled by the equipped vehicle.

18. The vehicular driving assist system of claim 17 , wherein said rearward viewing camera comprises a rear backup camera, and wherein during a reversing maneuver of the equipped vehicle, said rear backup camera views at least a portion of a vehicle bumper of the equipped vehicle, and wherein the equipped vehicle comprises a trailer hitch disposed at a rear portion of the equipped vehicle, and wherein, when the driver of the equipped vehicle is maneuvering the equipped vehicle to hitch a trailer to the equipped vehicle, said control processes image data captured by said rear backup camera and outputs for display by said video display screen video images derived from image data captured by said rear backup camera.

19. The vehicular driving assist system of claim 18 , wherein, responsive to said control processing image data captured by said rear backup camera, at least one overlay overlays the displayed video images to guide the driver when hitching the trailer to the trailer hitch of the equipped vehicle.

20. The vehicular driving assist system of claim 19 , wherein the at least one overlay aids in guiding connection of the trailer hitch of the equipped vehicle to a trailer tongue of the trailer.

21. The vehicular driving assist system of claim 17 , wherein data processing at said control comprises fusion of provided captured image data and provided captured radar data.

22. The vehicular driving assist system of claim 17 , wherein said video display screen of said display device comprises a backlit thin film transistor LCD video display screen backlit by at least one white light emitting light emitting diode.

23. The vehicular driving assist system of claim 22 , wherein said display device is operable to display video images at said video display screen with a display intensity, as viewed by the driver of the equipped vehicle, greater than 400 candelas/sq. meter.

24. The vehicular driving assist system of claim 17 , wherein said video display screen comprises an organic light emitting diode (OLED) video display screen.

25. A vehicular driving assist system, said vehicular driving assist system comprising:

a rearward viewing camera disposed at a rear portion of a vehicle equipped with said vehicular driving assist system;

said rearward viewing camera having a field of view at least rearward of the equipped vehicle;

said rearward viewing camera capturing frames of image data during operation of the equipped vehicle;

a driver-side camera disposed at a driver side portion of the equipped vehicle and a passenger-side camera disposed at a passenger side portion of the equipped vehicle;

wherein said driver-side camera has at least a sideward field of view external of the equipped vehicle, and wherein said passenger-side camera has at least a sideward field of view external of the equipped vehicle;

wherein said driver-side camera captures frames of image data during operation of the equipped vehicle;

wherein said passenger-side camera captures frames of image data during operation of the equipped vehicle;

at least one radar sensor disposed at the equipped vehicle;

the at least one radar sensor having a field of sensing external of the equipped vehicle;

wherein the at least one radar sensor captures radar data during operation of the equipped vehicle;

a control disposed in the equipped vehicle, wherein said control comprises a data processor;

wherein image data captured by said rearward viewing camera is provided to said control, and wherein image data captured by said driver-side camera is provided to said control, and wherein image data captured by said passenger-side camera is provided to said control, and wherein radar data captured by the at least one radar sensor is provided to said control;

wherein captured image data provided to said control is processed at said control, and wherein captured radar data provided to said control is processed at said control;

wherein processing at said control of provided captured image data comprises use of an edge detection algorithm;

wherein, via processing at said control of provided captured image data and provided captured radar data, presence of a person exterior the equipped vehicle is determined;

wherein said control connects to a vehicle network of the equipped vehicle;

wherein data processing at said control comprises fusion of provided captured image data and provided captured radar data;

wherein, via processing at said control of provided captured image data and provided captured radar data, a potential hazard presented by rear-approaching traffic approaching the equipped vehicle in a traffic lane adjacent to a traffic lane being traveled by the equipped vehicle is determined;

wherein said control outputs video image data for display by a display device of the equipped vehicle, said display device comprising a video display screen that displays video images for viewing by a driver of the equipped vehicle during operation of the equipped vehicle; and

wherein said control processes provided captured image data and provided captured radar data to alert the driver to the determined potential hazard presented by the rear-approaching traffic in the traffic lane adjacent to the traffic lane being traveled by the equipped vehicle.

26. The vehicular driving assist system of claim 25 , wherein said rearward viewing camera comprises a rear backup camera, and wherein during a reversing maneuver of the equipped vehicle, said rear backup camera views at least a portion of a vehicle bumper of the equipped vehicle, and wherein the equipped vehicle comprises a trailer hitch disposed at a rear portion of the equipped vehicle, and wherein, when the driver of the equipped vehicle is maneuvering the equipped vehicle to hitch a trailer to the equipped vehicle, said control processes image data captured by said rear backup camera and outputs for display by said video display screen video images derived from image data captured by said rear backup camera.

27. The vehicular driving assist system of claim 26 , wherein, responsive to said control processing image data captured by said rear backup camera, at least one overlay overlays the displayed video images to guide the driver when hitching the trailer to the trailer hitch of the equipped vehicle.

28. The vehicular driving assist system of claim 27 , wherein the at least one overlay aids in guiding connection of the trailer hitch of the equipped vehicle to a trailer tongue of the trailer.

29. The vehicular driving assist system of claim 25 , wherein said video display screen of said display device comprises a backlit thin film transistor LCD video display screen backlit by at least one white light emitting light emitting diode.

30. The vehicular driving assist system of claim 29 , wherein said display device is operable to display video images at said video display screen with a display intensity, as viewed by the driver of the equipped vehicle, greater than 200 candelas/sq. meter.

31. The vehicular driving assist system of claim 25 , wherein said video display screen comprises an organic light emitting diode (OLED) video display screen.

Continuity (20)
Continuation 16946763 · Jul 6, 2020
Continuation 16785816 · Feb 10, 2020
Continuation 16429171 · Jun 3, 2019
Continuation 16203953 · Nov 29, 2018
Continuation 15790173 · Oct 23, 2017
Continuation 15262480 · Sep 12, 2016
Continuation 14935699 · Nov 9, 2015
Continuation 14641812 · Mar 9, 2015
Continuation 14551608 · Nov 24, 2014
Continuation 14293487 · Jun 2, 2014
Continuation 13285128 · Oct 31, 2011
Continuation 11226628 · Sep 14, 2005
Provisional Application 60692113 · Jun 20, 2005
Provisional Application 60677990 · May 5, 2005
Provisional Application 60653787 · Feb 17, 2005
Provisional Application 60642227 · Jan 7, 2005
Provisional Application 60638250 · Dec 21, 2004
Provisional Application 60624091 · Nov 1, 2004
Provisional Application 60609642 · Sep 14, 2004
Related Publication 20210023995A1 · Jan 28, 2021
Cited By (3)
US 12,209,389 US 12,258,025 US 12,516,506