IP Library Granted Patent US 11,787,408
Granted Patent B2
US 11,787,408 · App. 16/180,468 · Granted Oct 17, 2023

System and method for controlling vehicle based on condition of driver

Inventor: Sang Kyun Sim (Anyang-si, KR)
Assignee: HL Klemove Corp.
B60W30/18163A61B5/0077A61B5/1112A61B5/163A61B5/165A61B5/18A61B5/4809A61B5/6893B60K28/06B60Q1/52B60W10/188B60W10/20B60W30/10B60W30/143B60W30/182B60W40/08B60W50/10B60W50/14G05D1/0055G05D1/0088G06T7/73G06V20/588G06V20/597G06V40/166G06V40/193A61B5/1122A61B2560/0242A61B2562/04B60W2040/0818B60W2420/42B60W2540/26B60W2554/4041B60W2900/00G05D2201/0213G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,787,408
App. No.
16/180,468
Granted
Oct 17, 2023
Kind
B2
Abstract

A system and method for controlling a vehicle based on a driver status are disclosed. The vehicle control system includes various sensing devices (including a camera, a vehicle dynamics sensor, a vehicle around view monitoring (AVM) camera, a periphery surveillance sensor, and a navigation device) and an electronic control unit (ECU). The ECU may analyze a driver status through the driver's face and pupils recognized by output signals of the sensing devices. If the driver has no intention to drive the vehicle, the ECU may control the vehicle to stop on a road shoulder, resulting in guarantee of safer driving.

Claims (47)

1. A method for controlling a vehicle comprising:

acquiring, by a driver surveillance camera, images of a driver's face and a movement of the driver's pupils;

receiving, by a controller, the acquired images as driver monitoring information;

analyzing, by the controller, a driver status using the images of the driver's face and the movement of the driver's pupils;

determining whether the analyzed driver status is normal or abnormal;

upon determining that the analyzed driver status is abnormal, controlling, by the controller, the vehicle to move toward a road shoulder,

wherein the controlling the vehicle to move toward the road shoulder includes:

upon determining that the driver status is abnormal, warning the driver of careless driving before moving to the road shoulder,

upon determination that the driver has no intention to drive the vehicle irrespective of the warning of careless driving, determining whether the vehicle is able to move toward the road shoulder,

controlling the vehicle to move toward the road shoulder when the vehicle is able to move toward the road shoulder including, upon determining that a current position of the vehicle is not in a lane nearest to the road shoulder, controlling the vehicle to move from a current lane to the lane nearest to the road shoulder, and

controlling deceleration of the vehicle when the vehicle is unable to move toward the road shoulder; and

upon determining that the analyzed driver status is normal, analyzing the movement of the driver's pupils; and

controlling, by the controller, vehicle movement based on the movement of the driver's pupils,

wherein the analyzing of the movement of the driver's pupils comprises filtering out meaningless movement irrelevant to vehicle control from among movements of the driver's pupil, and

the filtering out meaningless movement of the driver's pupils comprises:

obtaining current gaze position information of the driver acquired by the driver surveillance camera;

detecting a presence of any object within regions gazed at by the driver using information acquired by a vehicle around view monitoring (AVM) camera based on the current gaze position information; and

upon determining that the detected object is not a vehicle lane or road, determining the movement of the driver's pupils to be the meaningless movement.

2. The method according to claim 1 , wherein the controlling the vehicle to move toward the road shoulder further includes:

detecting a presence or absence of the road shoulder in which the vehicle is able to enter;

determining whether the current position of the vehicle is in the lane nearest to the detected road shoulder;

determining whether the road shoulder has a sufficiently large width in which the vehicle is able to stop on the road shoulder;

detecting a presence or absence of any other vehicle traveling on the road shoulder;

upon determining that the any other vehicle traveling on the road shoulder is not detected, calculating a vehicle stop point on the road shoulder; and

controlling the vehicle to move toward to stop at the calculated stop point of the road shoulder.

3. The method according to claim 2 , wherein the detecting a presence or absence of the road shoulder in which the vehicle is able to enter includes:

upon receiving, from a navigation device, at least one of information about the presence of the road shoulder in a forward region of the vehicle, information about horizontal and vertical distances from the vehicle to a start point of the road shoulder, or information about horizontal and vertical distances from the vehicle to an end point of the road shoulder, detecting the road shoulder based on the received information; or

if a solid line and a road edge at a right side of the vehicle are recognized through the vehicle around view monitoring (AVM) camera, detecting the road shoulder based on the recognized information.

4. The method according to claim 2 , wherein the determining whether the current position of the vehicle is in the lane nearest to the road shoulder includes:

receiving a horizontal distance from the vehicle to a start point of the road shoulder from a navigation device; and

if the received horizontal distance from the vehicle to the start point of the road shoulder is equal to or shorter than a predetermined distance, determining that the current position of the vehicle is in the lane nearest to the road shoulder.

5. The method according to claim 2 , wherein the calculating the vehicle stop point on the road shoulder includes calculating the vehicle stop point on the shoulder using a distance from the vehicle to an end point of the shoulder or using an end point of a road edge.

6. The method according to claim 1 , wherein the controlling the vehicle to move from the current lane to the lane nearest to the road shoulder includes:

recognizing information about objects located in backward and surrounding regions of the vehicle through a vehicle dynamics sensor, the vehicle around view monitoring (AVM) camera, and a periphery surveillance sensor, and determining whether lane change of the vehicle is available based on the recognized information; and

upon determining that the lane change of the vehicle is available, controlling the vehicle to move from the current lane to the lane nearest to the road shoulder.

7. The method according to claim 1 , further comprising at least one of:

turning on an emergency light; or

transmitting vehicle information, driver information, and driver status information to a central control center.

8. The method according to claim 1 , wherein the analyzing of the movement of the driver's pupils further comprises:

performing calibration on an image of the driver's pupils; and

detecting movement of the driver's pupils requesting control of vehicle movement from among the movements of the driver's pupils.

9. The method according to claim 8 , wherein the performing calibration on an image of the driver's pupils includes:

when the driver gazes at a left lane with respect to the current lane, storing position information of the driver's pupils captured by the driver surveillance camera;

when the driver looks forward in the current lane, storing position information of the driver's pupils captured by the driver surveillance camera; and

when the driver gazes at a right lane with respect to the current lane, storing position information of the driver's pupils captured by the driver surveillance camera.

10. The method according to claim 8 , wherein the detecting movement of the driver's pupils requesting control of the vehicle movement includes, upon determining that the movement of the driver's pupils is not identical to the meaningless movement, a vehicle lane is detected in regions gazed at by the driver, and the driver gazes at a vehicle lane region, in which the vehicle is entering, for a predetermined period of time, determining occurrence of the movement of the driver's pupils requesting control of vehicle movement.

11. The method according to claim 8 , wherein the detecting movement of the driver's pupils requesting control of vehicle movement includes, if dilation of the driver's pupils occurs, determining the occurrence of movement of the driver's pupils requesting control of vehicle movement.

Assignments (3)
MERGER Recorded Aug 15, 2022
From: MANDO MOBILITY SOLUTIONS CORPORATION
To: HL KLEMOVE CORP.
Reel/Frame 061148/0166 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2021
From: MANDO CORPORATION
To: MANDO MOBILITY SOLUTIONS CORPORATION
Reel/Frame 058598/0480 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 5, 2018
From: SIM, SANG KYUN
To: MANDO CORPORATION
Reel/Frame 047411/0199 →
Priority Claims (1)
KR 10-2017-0146215 · Nov 3, 2017 · national
Continuity (1)
Related Publication 20190135291A1 · May 9, 2019
Cited By (2)
US 12,208,794 US 12,485,899