IP Library Granted Patent US 12,485,883
Granted Patent B2
US 12,485,883 · App. 18/104,048 · Granted Dec 2, 2025

Systems and methods for navigating a vehicle

Inventors: Shai Shalev-Shwartz (Jeruslem, IL); Shaked Shammah (Jerusalem, IL); Amnon Shashua (Mevaseret Zion, IL)
Assignee: MOBILEYE VISION TECHNOLOGIES LTD.
B60W30/09B60W10/06B60W10/18B60W10/20B60W30/0953B60W30/0956B60W30/165B60W30/18154B60W30/18163B60W40/06B60W40/105B60W50/087B60W50/12B60W2420/403B60W2420/408B60W2510/18B60W2520/10B60W2520/105B60W2552/00B60W2554/00B60W2554/80B60W2555/20B60W2710/18B60W2710/20B60W2754/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,485,883
App. No.
18/104,048
Filed
Jan 31, 2023
Granted
Dec 2, 2025
Kind
B2
Art Unit
3666
USPC
701/41
Abstract

An autonomous system may selectively displace human driver control of a host vehicle. The system may receive an image representative of an environment of the host vehicle and detect an obstacle in the environment of the host vehicle based on analysis of the image. The system may monitor a driver input to a throttle, brake, and/or steering control associated with the host vehicle. The system may determine whether the driver input would result in the host vehicle navigating within a proximity buffer relative to the obstacle. If the driver input would not result in the host vehicle navigating within the proximity buffer, the system may allow the driver input to cause a corresponding change in one or more host vehicle motion control systems. If the driver input would result in the host vehicle navigating within the proximity buffer, the system may prevent the driver input from causing the corresponding change.

Claims (61)

1 . A system for selectively displacing human driver control of a host vehicle, the system comprising:

at least one processor comprising circuitry and a memory, wherein the memory incudes instructions that when executed by the circuitry cause the at least one processor to:

receive, from an image capture device, at least one image representative of an environment of the host vehicle;

detect at least one obstacle in the environment of the host vehicle based on an analysis of the at least one image;

monitor a driver input to at least one of a throttle control, a brake control, or a steering control associated with the host vehicle;

determine whether the driver input would result in the host vehicle navigating within a proximity buffer relative to the at least one obstacle;

allow the driver input to cause a corresponding change in one or more host vehicle motion control systems, if the at least one processing device determines that the driver input would not result in the host vehicle navigating within the proximity buffer relative to the at least one obstacle; and

prevent the driver input from causing the corresponding change in the one or more host vehicle motion control systems, if the at least one processing device determines that the driver input would result in the host vehicle navigating within the proximity buffer relative to the at least one obstacle,

wherein the at least one obstacle includes a target vehicle, and the proximity buffer relative to the target vehicle is determined based on a sum of at least:

a host vehicle acceleration distance, determined as a distance over which the host vehicle will travel if accelerated at an assumed acceleration rate of the host vehicle over a reaction time associated with the host vehicle,

a host vehicle stopping distance, determined as a distance required to reduce a determined speed of the host vehicle to zero at an assumed braking rate of the host vehicle, the determined speed corresponding to a speed of the host vehicle if the host vehicle accelerated at the assumed acceleration rate of the host vehicle over the reaction time associated with the host vehicle, and

a target vehicle stopping distance, determined as a distance required to reduce a current speed of the target vehicle to zero at an assumed braking rate of the target vehicle.

2 . The system of claim 1 , wherein the assumed acceleration rate of the host vehicle is a maximum acceleration capability of the host vehicle.

3 . The system of claim 1 , wherein the assumed acceleration rate of the host vehicle is a predetermined sub-maximal acceleration rate that is less than a maximum acceleration capability of the host vehicle.

4 . The system of claim 1 , wherein the assumed braking rate of the host vehicle is a maximum braking capability of the host vehicle.

5 . The system of claim 1 , wherein the assumed braking rate of the host vehicle is a predetermined sub-maximal braking rate that is less than a maximum braking capability of the host vehicle.

6 . The system of claim 1 , wherein the assumed braking rate of the target vehicle is determined based on analysis of the at least one image.

7 . The system of claim 6 , wherein the assumed braking rate of the target vehicle is determined based on a size of the target vehicle determined based on the analysis of the at least one image.

8 . The system of claim 6 , wherein the assumed braking rate of the target vehicle is determined based on a sensed condition of a road surface determined based on the analysis of the at least one image.

9 . The system of claim 1 , wherein the target vehicle is determined to be forward of the host vehicle, and wherein determining whether the driver input would result in the host vehicle navigating within the proximity buffer includes determining that the driver input would result in a change in longitudinal distance between the target vehicle and the host vehicle.

10 . The system of claim 1 , wherein the target vehicle is determined to be in a lane different from the host vehicle, and wherein determining whether the driver input would result in the host vehicle navigating within the proximity buffer includes determining that the driver input would result in a lateral movement of the host vehicle, such that after the lateral movement, the target vehicle will be forward of the host vehicle.

11 . The system of claim 1 , wherein the target vehicle is determined to be in a lane different from the host vehicle, and wherein determining whether the driver input would result in the host vehicle navigating within the proximity buffer includes determining that the driver input would result in a lateral movement of the host vehicle, such that after the lateral movement, the host vehicle will be forward of the target vehicle.

12 . The system of claim 1 , wherein the sum further includes a predetermined minimum distance to be maintained between the host vehicle and the target vehicle.

13 . The system of claim 1 , wherein the throttle control includes an accelerator pedal, the brake control includes a brake pedal, and the steering control includes a steering wheel, and wherein the driver input includes at least one of a depression of the accelerator pedal, a depression of the brake pedal, a lack of depression of the brake pedal, a rotation of the steering wheel, or a non-rotation of the steering wheel.

14 . The system of claim 13 , wherein the one or more host vehicle control systems include at least one steering actuator to control a heading of the host vehicle, a braking actuator to cause application of a host vehicle braking device, or an acceleration actuator to cause application of a host vehicle throttle.

15 . The system of claim 14 , wherein prevention of the driver input from causing the corresponding change in the one or more host vehicle motion control systems includes at least one of preventing driver input to the steering wheel from resulting in a corresponding response by the at least one steering actuator, preventing driver input to the brake pedal from resulting in a corresponding response by the braking actuator, or preventing driver input to the accelerator pedal from resulting in a corresponding response by the acceleration actuator.

16 . The system of claim 1 , wherein preventing the driver input from causing the corresponding change in the one or more host vehicle motion control systems includes preventing a motion of at least one of the throttle control, the brake control, or the steering control.

17 . The system of claim 1 , wherein preventing the driver input from causing the corresponding change in the one or more host vehicle motion control systems includes disabling operability of at least one of the throttle control, the brake control, or the steering control.

18 . The system of claim 1 , wherein the at least one processing device is configured to control navigation of the host vehicle autonomously during intervals in which the at least one processing device prevents the driver input from causing the corresponding change in the one or more host vehicle motion control systems.

19 . A method for selectively displacing human driver control of a host vehicle, the method comprising:

receiving, from an image capture device, at least one image representative of an environment of the host vehicle;

detecting at least one obstacle in the environment of the host vehicle based on an analysis of the at least one image;

monitoring a driver input to at least one of a throttle control, a brake control, or a steering control associated with the host vehicle;

determining whether the driver input would result in the host vehicle navigating within a proximity buffer relative to the at least one obstacle;

allowing the driver input to cause a corresponding change in one or more host vehicle motion control systems, if the at least one processing device determines that the driver input would not result in the host vehicle navigating within the proximity buffer relative to the at least one obstacle; and

preventing the driver input from causing the corresponding change in the one or more host vehicle motion control systems, if the at least one processing device determines that the driver input would result in the host vehicle navigating within the proximity buffer relative to the at least one obstacle,

wherein the at least one obstacle includes a target vehicle, and the proximity buffer relative to the target vehicle is determined based on a sum of at least:

a host vehicle acceleration distance, determined as a distance over which the host vehicle will travel if accelerated at a maximum acceleration capability of the host vehicle over a reaction time associated with the host vehicle,

a host vehicle stopping distance, determined as a distance required to reduce a determined speed of the host vehicle to zero at a maximum braking rate capability of the host vehicle, the determined speed corresponding to a speed of the host vehicle if the host vehicle accelerated at the maximum acceleration capability of the host vehicle over the reaction time associated with the host vehicle, and

a target vehicle stopping distance, determined as a distance required to reduce a current speed of the target vehicle to zero at an assumed maximum braking rate capability of the target vehicle.

20 . The method of claim 19 , wherein the assumed acceleration rate of the host vehicle is a maximum acceleration capability of the host vehicle.

21 . The method of claim 19 , wherein the assumed acceleration rate of the host vehicle is a predetermined sub-maximal acceleration rate that is less than a maximum acceleration capability of the host vehicle.

22 . The method of claim 19 , wherein the assumed braking rate of the host vehicle is a maximum braking capability of the host vehicle.

23 . The method of claim 19 , wherein the assumed braking rate of the host vehicle is a predetermined sub-maximal braking rate that is less than a maximum braking capability of the host vehicle.

24 . The method of claim 19 , wherein the assumed braking rate of the target vehicle is determined based on at least one of a size of the target vehicle or a sensed condition of a road surface determined based on analysis of the at least one image.

25 . The method of claim 19 , wherein the target vehicle is determined to be forward of the host vehicle, and wherein determining whether the driver input would result in the host vehicle navigating within the proximity buffer includes determining that the driver input would result in a change in longitudinal distance between the target vehicle and the host vehicle.

26 . The method of claim 19 , wherein the target vehicle is determined to be in a lane different from the host vehicle, and wherein determining whether the driver input would result in the host vehicle navigating within the proximity buffer includes determining that the driver input would result in a lateral movement of the host vehicle, such that after the lateral movement, the target vehicle will be in the same lane as the host vehicle.

27 . A non-transitory computer readable medium containing instructions that when executed by at least one processor, cause the at least one processor to perform a method for selectively displacing human driver control of a host vehicle, the method comprising:

receiving, from an image capture device, at least one image representative of an environment of the host vehicle;

detecting at least one obstacle in the environment of the host vehicle based on an analysis of the at least one image;

monitoring a driver input to at least one of a throttle control, a brake control, or a steering control associated with the host vehicle;

determining whether the driver input would result in the host vehicle navigating within a proximity buffer relative to the at least one obstacle;

allowing the driver input to cause a corresponding change in one or more host vehicle motion control systems, if the at least one processing device determines that the driver input would not result in the host vehicle navigating within the proximity buffer relative to the at least one obstacle; and

preventing the driver input from causing the corresponding change in the one or more host vehicle motion control systems, if the at least one processing device determines that the driver input would result in the host vehicle navigating within the proximity buffer relative to the at least one obstacle,

wherein the at least one obstacle includes a target vehicle, and the proximity buffer relative to the target vehicle is determined based on a sum of at least:

a host vehicle acceleration distance, determined as a distance over which the host vehicle will travel if accelerated at a maximum acceleration capability of the host vehicle over a reaction time associated with the host vehicle,

a host vehicle stopping distance, determined as a distance required to reduce a determined speed of the host vehicle to zero at a maximum braking rate capability of the host vehicle, the determined speed corresponding to a speed of the host vehicle if the host vehicle accelerated at the maximum acceleration capability of the host vehicle over the reaction time associated with the host vehicle, and

a target vehicle stopping distance, determined as a distance required to reduce a current speed of the target vehicle to zero at an assumed maximum braking rate capability of the target vehicle.

28 . The non-transitory computer readable medium of claim 27 , wherein the assumed braking rate of the target vehicle is determined based on analysis of the at least one image.

29 . The non-transitory computer readable medium of claim 28 , wherein the assumed braking rate of the target vehicle is determined based on a size of the target vehicle determined based on the analysis of the at least one image.

30 . The non-transitory computer readable medium of claim 28 , wherein the assumed braking rate of the target vehicle is determined based on a sensed condition of a road surface determined based on the analysis of the at least one image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 31, 2023
From: SHALEV-SHWARTZ, SHAI; SHAMMAH, SHAKED; SHASHUA, AMNON
To: LTD., MOBILEYE VISION TECHNOLOGIES
Reel/Frame 062552/0350 →
Continuity (10)
Continuation 17182702 · Feb 23, 2021
Continuation 16429901 · Jun 3, 2019
Continuation 16359488 · Mar 20, 2019
Provisional Application 62777914 · Dec 11, 2018
Provisional Application 62772366 · Nov 28, 2018
Provisional Application 62724355 · Aug 29, 2018
Provisional Application 62718554 · Aug 14, 2018
Provisional Application 62646579 · Mar 22, 2018
Provisional Application 62645479 · Mar 20, 2018
Related Publication 20230166729A1 · Jun 1, 2023
References Cited (93)
US 5375059A · Kyrtsos · 1994 [cited by applicant]
US 5555503A · Kyrtsos · 1996 [cited by applicant]
US 6737963B2 · Gutta · 2004 [cited by applicant]
US 7124027B1 · Ernst, Jr. · 2006 [cited by applicant]
US 7164117B2 · Breed · 2007 [cited by applicant]
US 7243945B2 · Breed · 2007 [cited by applicant]
US 7415126B2 · Breed · 2008 [cited by applicant]
US 7663502B2 · Breed · 2010 [cited by applicant]
US 7941261B2 · Johnsen · 2011 [cited by applicant]
US 8849494B1 · Herbach · 2014 [cited by examiner]
US 8884782B2 · Rubin · 2014 [cited by applicant]
US 8996224B1 · Herbach · 2015 [cited by applicant]
US 9020697B2 · Ricci · 2015 [cited by applicant]
US 9082239B2 · Ricci · 2015 [cited by applicant]
US 9587952B1 · Slusar · 2017 [cited by examiner]
US 9767366B1 · Fairfield · 2017 [cited by applicant]
US 9849852B1 · Larner · 2017 [cited by examiner]
US 9934440B1 · Kim · 2018 [cited by applicant]
US 9950722B2 · Kumar · 2018 [cited by applicant]
US 9953535B1 · Canavor · 2018 [cited by examiner]
US 10421453B1 · Ferguson · 2019 [cited by examiner]
US 20030195703A1 · Ibrahim · 2003 [cited by applicant]
US 20060293856A1 · Foessel et al. · 2006 [cited by applicant]
US 20070106475A1 · Kondoh · 2007 [cited by applicant]
US 20080243389A1 · Inoue et al. · 2008 [cited by applicant]
US 20090295601A1 · Villaume · 2009 [cited by applicant]
US 20090322500A1 · Chatterjee et al. · 2009 [cited by applicant]
US 20100256852A1 · Mudalige · 2010 [cited by applicant]
US 20120078498A1 · Iwasaki et al. · 2012 [cited by applicant]
US 20130110368A1 · Zagorski · 2013 [cited by examiner]
US 20130274958A1 · Uno · 2013 [cited by applicant]
US 20140032023A1 · Kumar · 2014 [cited by applicant]
US 20150158495A1 · Duncan · 2015 [cited by applicant]
US 20150161893A1 · Duncan · 2015 [cited by applicant]
US 20150161894A1 · Duncan · 2015 [cited by examiner]
US 20150234045A1 · Rosenblum · 2015 [cited by applicant]
US 20160016566A1 · Kumar · 2016 [cited by applicant]
US 20160075332A1 · Edo-Ros · 2016 [cited by applicant]
US 20160082885A1 · Rashid · 2016 [cited by applicant]
US 20160170021A1 · Rashid · 2016 [cited by applicant]
US 20160172842A1 · Panosyan · 2016 [cited by applicant]
US 20170057520A1 · Letwin · 2017 [cited by applicant]
US 20170101091A1 · Nilsson et al. · 2017 [cited by applicant]
US 20170220876A1 · Gao · 2017 [cited by applicant]
US 20180001893A1 · Wei et al. · 2018 [cited by applicant]
US 20180029592A1 · Trombley et al. · 2018 [cited by applicant]
US 20180297567A1 · Gonidec · 2018 [cited by applicant]
US 20180307925A1 · Wisniowski · 2018 [cited by applicant]
US 20180321686A1 · Kanzawa · 2018 [cited by applicant]
US 20180329035A1 · Pacala · 2018 [cited by applicant]
US 20190011556A1 · Pacala · 2019 [cited by applicant]
US 20190092318A1 · Mei · 2019 [cited by applicant]
US 20190156485A1 · Pfeiffer · 2019 [cited by applicant]
US 20190179028A1 · Pacala · 2019 [cited by applicant]
CN 104228836A · 2014 [cited by applicant]
CN 104837707A · 2015 [cited by applicant]
CN 107000746A · 2017 [cited by applicant]
CN 107650907A · 2018 [cited by applicant]
DE 102014215243A1 · 2016 [cited by applicant]
EP 1898232A1 · 2008 [cited by applicant]
JP H08310359A · 2004 [cited by applicant]
JP 2013206297 · 2013 [cited by applicant]
KR 101703144B1 · 2017 [cited by applicant]
KR 101728406B1 · 2017 [cited by applicant]
KR 1020170054186A · 2017 [cited by applicant]
KR 1020170126909A · 2017 [cited by applicant]
KR 1020180008593A · 2018 [cited by applicant]
WO WO2016130719A2 · 2016 [cited by applicant]
WO WO2017195520A1 · 2017 [cited by applicant]
WO WO2018015811A1 · 2018 [cited by applicant]
Eshed, et al., On surveillance for safety critical events: In-vehicle video networks for predictive driver assistance systems (Year: 2015). [cited by examiner]
Office Action issued in Chinese Patent Application No. 202211011663.4, dated Sep. 6, 2024, and translation thereof (14 pages). [cited by applicant]
Richard Bellman, “Mathematical Theory of Control Processes,” Departments of Mathematics and Electrical Engineering, University of Southern California, vol. II Nonlinear Process, 1971, Academic Press, New York and London… [cited by applicant]
Shai Shalev-Shwartz, et al., “On a Formal Model of Safety and Scalable Self-Driving Cars,” Aug. 21, 2017, available at https://arxiv.org/abs/1708.06374. [cited by applicant]
IIHS, GM Front Crash prevention systems cut police-reported crashes, Nov. 13, 2018, IHSS News, available at https://www.iihs.org/iihs/news/desktopnews/gm-front-crash-prevention-systems-cut-police-reported-crashes. [cited by applicant]
Shai Shalev-Shwartz, et al., “Vision Zero: on a Provable Method for Eliminating Roadway Accidents without Compromising Traffic Throughput,” Dec. 9, 2018, available at https://arxiv.org/abs/1901.05022v2. [cited by applicant]
Ratings Group Report: “Euro NCAP Rating Review 2015,” Mar. 2015, available at https://cdn.euroncap.com/media/16470/ratings-group-report-2015-version-10-with-appendix.pdf. [cited by applicant]
Paul Seredynski, “As Early Autonomy Questions Roil the Industry, Toyota Hews to Driver-Enhancement Philosophy,” Jun. 11, 2018, available at https://www.sae.org/news/2018/06/tu-auto-interview-with-toyotas-ryan-eustice. [cited by applicant]
Leemon C Baird, “Reinforcement Learning in Continuous Time: Advantage Updating in Neural Networks,” IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on Neural Networks, vol. 4, pp. … [cited by applicant]
Richard Bellman, “Dynamic Programming and Lagrange Multipliers,” Proceedings of the National Academy of Sciences of the United States of America, 42(10): 767, 1956. [cited by applicant]
Peng Lin, et al., Discrete Element Crowd Model for Pedestrian Evacuation Through an Exit, Chinese Physics B, 25(3):034501, Oct. 24, 2015, available at https://arxiv.org/abs/1510.07126. [cited by applicant]
Volodymyr Mnih, et al., “Human-Level Control Through Deep Reinforcement Learning,” Nature, 518(7540):529-533, Feb. 25, 2015, available at https://web.stanford.edu/class/psych209/Readings/MnihEtAlHassibis15NatureControlD… [cited by applicant]
Shai Shalev-Shwartz, et al., “Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving,” Oct. 11, 2016, available at https://arxiv.org/abs/1610.03295. [cited by applicant]
Richard S Sutton, et al., “Between MDPs and semi-MDPs: A framework for Temporal Abstraction in Reinforcement Learning.” Artificial intelligence, 112(1):181-211, Dec. 1, 1998, available at http://www-anw.cs.umass.edu/˜ba… [cited by applicant]
L.G. Valiant, “A Theory of the Learnable,” Communications of the ACM, 27(11): 1134-1142 Nov. 1984, available at https://people.mpi-inf.mpg.de/˜mehlhorn/SeminarEvolvability/ValiantLearnable.pdf. [cited by applicant]
Richard Bellman, “Introduction to the Mathematical Theory of Control Processes, vol. 2.” IMA, 1971. [cited by applicant]
International Search Report and Written Opinion in PCT/IB2019/000281, mailed Jun. 3, 2020 (8 pages). [cited by applicant]
Search Report from Korean Intellectual Property Office mailed Oct. 25, 2019 for Korean application No. 10-2019-7030686 (7 pages). [cited by applicant]
Eshed, Ohn-Bar, On surveillance for safety critical events: In-vehicle video networks for predictive driver assistance systems, Computer Vision and Image Understanding, Science Direct, vol. 134, 2015 (Year: 2015). [cited by applicant]
Office Action issued in Japanese Patent Application No. 2022-074527, dated Aug. 16, 2022 (5 pages). [cited by applicant]
Office Action issued in Japanese Patent Application No. 2022-074462, dated Aug. 16, 2022 (9 pages). [cited by applicant]
Office Action issued in Chinese Patent Application No. 201980002445.4, dated Sep. 1, 2022 and English translation thereof (34 pages). [cited by applicant]
Office Action issued in Chinese Patent Application No. 202110235271.5, dated Dec. 18, 2023, and translation thereof (25 pages). [cited by applicant]