IP Library › Granted Patent US 12,311,930
Granted Patent B2
US 12,311,930 · App. 17/811,312 · Granted May 27, 2025

Vehicular driving assistance system with enhanced traffic lane determination

Inventors: Alan M. Cordeiro (Farmington Hills, MI); Tejas Murlidhar Varunjikar (Troy, MI); Mayank Hitesh Desai (Novi, MI); Harold E. Joseph (Brampton, CA)
Assignee: Magna Electronics Inc.
B60W30/12B60W10/20B60W50/0205B60W50/029G06V20/588B60W2050/0215B60W2050/0292B60W2420/403B60W2510/202B60W2540/18B60W2552/53B60W2556/20B60W2556/35
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,311,930
App. No.
17/811,312
Granted
May 27, 2025
Kind
B2
Abstract

A vehicular driver assistance system includes a front camera module (FCM) disposed at a vehicle. The system, responsive to processing captured image data, generates FCM lane information including information regarding a traffic lane the vehicle is currently traveling along. An e-Horizon module (EHM) generates EHM lane information including information regarding the traffic lane the vehicle is currently traveling along. The vehicular driver assistance system determines an FCM correlation using the FCM lane information and sensor data captured by at least one exterior sensor. The vehicular driver assistance system determines an EHM correlation using the EHM lane information and the sensor data captured by the at least one exterior sensor. Responsive to determining the FCM correlation and the EHM correlation, the system controls lateral movement of the vehicle based on one selected from the group consisting of (i) the FCM lane information and (ii) the EHM lane information.

Claims (48)

1. A vehicular driver assistance system, the vehicular driver assistance system comprising:

a front camera module (FCM) disposed at a vehicle equipped with the vehicular driver assistance system and viewing exterior of the equipped vehicle, the FCM comprising a camera, the camera capturing image data;

an image processor for processing image data captured by the camera, wherein the vehicular driver assistance system, responsive to processing at the image processor of image data captured by the camera, generates FCM lane information comprising information regarding a traffic lane along which the equipped vehicle is traveling;

an e-Horizon module (EHM) comprising one or more localization sensors, wherein the EHM uses map data to anticipate a road ahead of the equipped vehicle, and wherein the EHM generates EHM lane information comprising information regarding the traffic lane along which the equipped vehicle is traveling;

at least one secondary sensor disposed exterior of the equipped vehicle and at least partially sensing the traffic lane along which the equipped vehicle is traveling, the at least one secondary sensor capturing secondary sensor data;

wherein the vehicular driver assistance system determines an FCM correlation using the generated FCM lane information and the secondary sensor data captured by the at least one secondary sensor, the FCM correlation indicating a difference between the generated FCM lane information and the secondary sensor data;

wherein the vehicular driver assistance system determines an EHM correlation using the generated EHM lane information and the secondary sensor data captured by the at least one secondary sensor, the EHM correlation indicating a difference between the generated EHM lane information and the secondary sensor data;

wherein, responsive to the FCM correlation satisfying an FCM correlation threshold, the vehicular driver assistance system controls lateral movement of the equipped vehicle based at least in part on the generated FCM lane information; and

wherein, responsive to the EHM correlation satisfying an EHM correlation threshold, the vehicular driver assistance system controls lateral movement of the equipped vehicle based at least in part on the generated EHM lane information.

2. The vehicular driver assistance system of claim 1 , wherein the vehicular driver assistance system controls the lateral movement of the equipped vehicle by generating at least one selected from the group consisting of (i) a steering angle command and (ii) a steering torque command.

3. The vehicular driver assistance system of claim 1 , wherein the FCM outputs an FCM status signal indicating a validity of the generated FCM lane information, and wherein the vehicular driver assistance system, responsive to determining that that the FCM status signal indicates the generated FCM lane information is invalid, controls lateral movement of the equipped vehicle based on the generated EHM lane information.

4. The vehicular driver assistance system of claim 1 , wherein the EHM outputs an EHM status signal indicating a validity of the generated EHM lane information, and wherein the vehicular driver assistance system, responsive to determining that that the EHM status signal indicates the generated EHM lane information is invalid, controls lateral movement of the equipped vehicle based on the generated FCM lane information.

5. The vehicular driver assistance system of claim 1 , wherein the vehicular driver assistance system, responsive to the EHM correlation failing to satisfy the EHM correlation threshold, controls lateral movement of the equipped vehicle using the generated FCM lane information.

6. The vehicular driver assistance system of claim 1 , wherein the vehicular driver assistance system, responsive to determining that the FCM correlation failing to satisfy the FCM correlation threshold, controls lateral movement of the equipped vehicle using the generated EHM lane information.

7. The vehicular driver assistance system of claim 1 , wherein the vehicular driver assistance system determines, based on the generated FCM lane information, a first distance between the equipped vehicle and a lane marker of the traffic lane along which the equipped vehicle is traveling, and wherein the vehicular driver assistance system determines, based on the secondary sensor data captured by the at least one secondary sensor, a second distance between the equipped vehicle and the lane marker of the traffic lane along which the equipped vehicle is traveling, and wherein the FCM correlation is based on a difference between the first distance and the second distance.

8. The vehicular driver assistance system of claim 1 , wherein the vehicular driver assistance system determines, based on the generated EHM lane information, a first distance between the equipped vehicle and a lane marker of the traffic lane along which the equipped vehicle is traveling, and wherein the vehicular driver assistance system determines, based on the secondary sensor data captured by the at least one secondary sensor, a second distance between the equipped vehicle and the lane marker of the traffic lane along which the equipped vehicle is traveling, and wherein the EHM correlation is based on a difference between the first distance and the second distance.

9. The vehicular driver assistance system of claim 1 , wherein the vehicular driver assistance system determines the FCM correlation and the EHM correlation in response to determining a difference between the generated FCM lane information and the generated EHM lane information exceeds a threshold.

10. The vehicular driver assistance system of claim 1 , wherein the at least one secondary sensor comprises a second camera.

11. The vehicular driver assistance system of claim 1 , wherein the at least one secondary sensor comprises at least one camera of a surround view camera (SVC) system of the equipped vehicle.

12. The vehicular driver assistance system of claim 1 , comprising an electronic control unit (ECU) comprising electronic circuitry and associated software.

13. The vehicular driver assistance system of claim 12 , wherein the electronic circuitry includes a data processor for processing secondary sensor data captured by the at least one secondary sensor, and wherein the vehicular driver assistance system determines the FCM correlation using the generated FCM lane information and via processing at the ECU of the secondary sensor data captured by the at least one secondary sensor, and wherein the vehicular driver assistance system determines the EHM correlation using the generated EHM lane information and via processing at the ECU of the secondary sensor data captured by the at least one secondary sensor.

14. The vehicular driver assistance system of claim 12 , wherein the electronic circuitry of the ECU includes the image processor.

15. The vehicular driver assistance system of claim 1 , wherein the FCM comprises the image processor.

16. A vehicular driver assistance system, the vehicular driver assistance system comprising:

a front camera module (FCM) disposed at a vehicle equipped with the vehicular driver assistance system and viewing exterior of the equipped vehicle, the FCM comprising a camera, the camera capturing image data;

an image processor for processing image data captured by the camera, wherein the vehicular driver assistance system, responsive to processing at the image processor of image data captured by the camera, generates FCM lane information comprising information regarding a traffic lane along which the equipped vehicle is traveling;

an e-Horizon module (EHM) comprising one or more localization sensors, wherein the EHM uses map data to anticipate a road ahead of the equipped vehicle, and wherein the EHM generates EHM lane information comprising information regarding the traffic lane along which the equipped vehicle is traveling;

at least one secondary sensor comprising a surround view camera (SCV) disposed at an exterior side mirror of the equipped vehicle and at least partially sensing the traffic lane along which the equipped vehicle is traveling, the at least one secondary sensor capturing secondary sensor data;

wherein the vehicular driver assistance system determines an FCM correlation using the generated FCM lane information and the secondary sensor data captured by the at least one secondary sensor, the FCM correlation indicating a difference between the generated FCM lane information and the secondary sensor data;

wherein the vehicular driver assistance system determines an EHM correlation using the generated EHM lane information and the secondary sensor data captured by the at least one secondary sensor, the EHM correlation indicating a difference between the generated EHM lane information and the secondary sensor data;

wherein, responsive to the FCM correlation failing to satisfy an FCM correlation threshold, the vehicular driver assistance system controls lateral movement of the equipped vehicle based on the generated EHM lane information; and

wherein, responsive to the EHM correlation failing to satisfy an EHM correlation threshold, the vehicular driver assistance system controls lateral movement of the equipped vehicle based on the generated FCM lane information.

17. The vehicular driver assistance system of claim 16 , wherein the vehicular driver assistance system controls the lateral movement of the equipped vehicle by generating at least one selected from the group consisting of (i) a steering angle command and (ii) a steering torque command.

18. The vehicular driver assistance system of claim 16 , wherein the vehicular driver assistance system determines, based on the generated FCM lane information, a first distance between the equipped vehicle and a lane marker of the traffic lane along which the equipped vehicle is traveling, and wherein the vehicular driver assistance system determines, based on the secondary sensor data captured by the at least one secondary sensor, a second distance between the equipped vehicle and the lane marker of the traffic lane along which the equipped vehicle is traveling, and wherein the FCM correlation is based on a difference between the first distance and the second distance.

19. The vehicular driver assistance system of claim 16 , wherein the vehicular driver assistance system determines, based on the generated EHM lane information, a first distance between the equipped vehicle and a lane marker of the traffic lane along which the equipped vehicle is traveling, and wherein the vehicular driver assistance system determines, based on the secondary sensor data captured by the at least one secondary sensor, a second distance between the equipped vehicle and the lane marker of the traffic lane along which the equipped vehicle is traveling, and wherein the EHM correlation is based on a difference between the first distance and the second distance.

20. A vehicular driver assistance system, the vehicular driver assistance system comprising:

a front camera module (FCM) disposed at a vehicle equipped with the vehicular driver assistance system and viewing exterior of the equipped vehicle, the FCM comprising a camera, the camera capturing image data;

an electronic control unit (ECU) comprising electronic circuitry and associated software, wherein the electronic circuitry comprises an image processor for processing image data captured by the camera, and wherein the vehicular driver assistance system, responsive to processing at the image processor of image data captured by the camera, generates FCM lane information comprising information regarding a traffic lane along which the equipped vehicle is traveling;

an e-Horizon module (EHM) comprising one or more localization sensors, wherein the EHM uses map data to anticipate a road ahead of the equipped vehicle, and wherein the EHM generates EHM lane information comprising information regarding the traffic lane along which the equipped vehicle is traveling;

a plurality of secondary sensors disposed exterior of the equipped vehicle and at least partially sensing the traffic lane along which the equipped vehicle is traveling, the plurality of secondary sensors capturing secondary sensor data;

wherein the vehicular driver assistance system determines an FCM correlation using the generated FCM lane information and the secondary sensor data captured by the plurality of secondary sensors, the FCM correlation indicating a difference between the generated FCM lane information and the secondary sensor data;

wherein the vehicular driver assistance system determines an EHM correlation using the generated EHM lane information and the secondary sensor data captured by the plurality of secondary sensors, the EHM correlation indicating a difference between the generated EHM lane information and the secondary sensor data;

wherein, responsive to the FCM correlation satisfying an FCM correlation threshold, the vehicular driver assistance system controls lateral movement of the equipped vehicle based at least in part on the generated FCM lane information; and

wherein, responsive to the EHM correlation satisfying an EHM correlation threshold, the vehicular driver assistance system controls lateral movement of the equipped vehicle based at least in part on the generated EHM lane information.

21. The vehicular driver assistance system of claim 20 , wherein the FCM outputs an FCM status signal indicating a validity of the generated FCM lane information, and wherein the vehicular driver assistance system, responsive to determining that that the FCM status signal indicates the generated FCM lane information is invalid, controls lateral movement of the equipped vehicle based on the generated EHM lane information.

22. The vehicular driver assistance system of claim 20 , wherein the EHM outputs an EHM status signal indicating a validity of the generated EHM lane information, and wherein the vehicular driver assistance system, responsive to determining that that the EHM status signal indicates the generated EHM lane information is invalid, controls lateral movement of the equipped vehicle based on the generated FCM lane information.

23. The vehicular driver assistance system of claim 20 , wherein the vehicular driver assistance system, responsive to the EHM correlation failing to satisfy the EHM correlation threshold, controls lateral movement of the equipped vehicle using the generated FCM lane information.

24. The vehicular driver assistance system of claim 20 wherein the vehicular driver assistance system, responsive to the FCM correlation failing to satisfy the FCM correlation threshold, controls lateral movement of the equipped vehicle using the generated EHM lane information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2025
From: CORDEIRO, ALAN M.; VARUNJIKAR, TEJAS MURLIDHAR; DESAI, MAYANK HITESH; JOSEPH, HAROLD E.
To: MAGNA ELECTRONICS INC.
Reel/Frame 070202/0636 →
Continuity (2)
Provisional Application 63203123 · Jul 9, 2021
Related Publication 20230008230A1 · Jan 12, 2023
References Cited (54)
US 5550677A · Schofield et al. · 1996 [cited by applicant]
US 5670935A · Schofield et al. · 1997 [cited by applicant]
US 5949331A · Schofield et al. · 1999 [cited by applicant]
US 6580986B1 · Uenuma et al. · 2003 [cited by applicant]
US 6690268B2 · Schofield et al. · 2004 [cited by applicant]
US 6824281B2 · Schofield et al. · 2004 [cited by applicant]
US 6882287B2 · Schofield · 2005 [cited by applicant]
US 7038577B2 · Pawlicki et al. · 2006 [cited by applicant]
US 7480149B2 · DeWard et al. · 2009 [cited by applicant]
US 7720580B2 · Higgins-Luthman · 2010 [cited by applicant]
US 7855755B2 · Weller et al. · 2010 [cited by applicant]
US 8256821B2 · Lawlor et al. · 2012 [cited by applicant]
US 9180908B2 · Van Dan Elzen et al. · 2015 [cited by applicant]
US 9340227B2 · Bajpai · 2016 [cited by applicant]
US 9487159B2 · Achenbach · 2016 [cited by applicant]
US 9596387B2 · Achenbach et al. · 2017 [cited by applicant]
US 9871971B2 · Wang et al. · 2018 [cited by applicant]
US 9896039B2 · Achenbach et al. · 2018 [cited by applicant]
US 9988047B2 · Johnson et al. · 2018 [cited by applicant]
US 10032369B2 · Koravadi · 2018 [cited by applicant]
US 10055651B2 · Chundrlik, Jr. et al. · 2018 [cited by applicant]
US 10071687B2 · Ihlenburg et al. · 2018 [cited by applicant]
US 10099614B2 · Diessner · 2018 [cited by applicant]
US 10268904B2 · Gupta · 2019 [cited by applicant]
US 10315651B2 · Fiaschetti et al. · 2019 [cited by applicant]
US 10449899B2 · Gupta et al. · 2019 [cited by applicant]
US 10583779B2 · Koravadi · 2020 [cited by applicant]
US 11608112B2 · Okouneva · 2023 [cited by examiner]
US 11648943B2 · Lee · 2023 [cited by examiner]
US 20090295181A1 · Lawlor et al. · 2009 [cited by applicant]
US 20110010054A1 · Wilson-Jones et al. · 2011 [cited by applicant]
US 20140160284A1 · Achenbach et al. · 2014 [cited by applicant]
US 20140226012A1 · Achenbach · 2014 [cited by applicant]
US 20150015713A1 · Wang et al. · 2015 [cited by applicant]
US 20150327398A1 · Achenbach et al. · 2015 [cited by applicant]
US 20160159394A1 · Ryu et al. · 2016 [cited by applicant]
US 20190315363A1 · Kim · 2019 [cited by examiner]
US 20200114933A1 · Ono · 2020 [cited by examiner]
US 20210316728A1 · Won · 2021 [cited by examiner]
US 20210354637A1 · Hwang · 2021 [cited by examiner]
US 20220048504A1 · Prasad Challa et al. · 2022 [cited by applicant]
US 20220048509A1 · Prasad Challa · 2022 [cited by applicant]
US 20220048566A1 · Prasad Challa et al. · 2022 [cited by applicant]
US 20220108117A1 · Potnis · 2022 [cited by applicant]
US 20220114374A1 · Khatri · 2022 [cited by applicant]
US 20220135030A1 · Varunjikar et al. · 2022 [cited by applicant]
US 20220176960A1 · Awathe et al. · 2022 [cited by applicant]
US 20220363250A1 · Varunjikar et al. · 2022 [cited by applicant]
US 20220410916A1 · Cordeiro et al. · 2022 [cited by applicant]
US 20230009269A1 · Prasad · 2023 [cited by examiner]
EP 3360746A1 · 2018 [cited by applicant]
Snider J.M., “Automatic Steering Methods for Autonomous Automobile Path Tracking”, Feb. 2009, CMU thesis. [cited by applicant]
Werling et al., Invariant Trajectory Tracking With a Full-Size Autonomous Road Vehicle, IEEE, vol. 26, No. 4, Aug. 2010. [cited by applicant]
Werling et al., Optimal trajectories for time-critical street scenarios using discretized terminal manifolds, The International Journal of Robotics Research, Mar. 2012. [cited by applicant]