IP Library Granted Patent US 10,696,224
Granted Patent B2
US 10,696,224 · App. 15/919,224 · Granted Jun 30, 2020

Driving notification method and driving notification system

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,696,224
App. No.
15/919,224
Granted
Jun 30, 2020
Kind
B2
Abstract

A driving notification method and a driving notification system are provided. The driving notification system is disposed on a mobile vehicle, and the driving notification method includes following steps. A surrounding image is captured through an image capturing device. A target object detection is performed to the surrounding image to obtain a target position of an image detection object. A gaze position of a driver is detected through an eye tracking device. Whether the gaze position matches the target position is determined. An alarm device is controlled to activate an alarming operation if the gaze position does not match the target position.

Claims (59)

1. A driving notification method, for a driving notification system comprising an eye tracking device and an image capturing device, the driving notification system being disposed on a mobile vehicle, the driving notification method comprising:

(a) capturing a surrounding image through the image capturing device;

(b) performing a target object detection to the surrounding image to obtain a target position of an image detection object;

(c) detecting a gaze position of a driver through the eye tracking device;

(d) determining whether the gaze position matches the target position; and

(e) controlling an alarm device to activate an alarming operation if the gaze position does not match the target position,

wherein the step (c) comprises:

obtaining a reference set value of a projection plane of the eye tracking device and a projection position reflecting an eye gaze of the driver on the projection plane, wherein the reference set value comprises a width and a height of the projection plane, and a distance between the projection plane and the mobile vehicle; and

converting the projection position on the projection plane into a relative position on the surrounding image to obtain the gaze position of the driver according to a field of view of the image capturing device and the reference set value of the projection plane,

wherein the step (d) comprises:

obtaining a specific block on the surrounding image by serving the gaze position of the driver as an extending basis, wherein a size of the specific block corresponds to a size of the image detection object; and

determining image similarity between the specific block and the image detection object,

determining that the gaze position matches the target position if the image similarity between the specific block and the image detection object is greater than a threshold; and

determining that the gaze position does not match the target position if the image similarity between the specific block and the image detection object is less than or equal to the threshold.

2. The driving notification method as claimed in claim 1 , wherein the driving notification method further comprises the following step after the step (d):

controlling the alarm device not to activate the alarming operation if the gaze position matches the target position.

3. The driving notification method as claimed in claim 1 , wherein the driving notification method further comprises the following steps after the step (e):

performing the step (a) to the step (d) repeatedly;

controlling the alarm device to stop performing the alarming operation if the gaze position matches the target position after performing the step (a) to the step (d) repeatedly; and

controlling a vehicle driving element of the mobile vehicle according to a target object corresponding to the image detection object if the gaze position does not match the target position after performing the step (a) to the step (d) repeatedly.

4. The driving notification method as claimed in claim 3 , wherein the target object comprises a traffic sign or an obstruction, and the step of controlling the vehicle driving element of the mobile vehicle according to the target object corresponding to the image detection object comprises:

controlling a speed control device of the mobile vehicle according to the traffic sign or the obstruction to lower a driving speed of the mobile vehicle.

5. The driving notification method as claimed in claim 1 , wherein the driving notification method further comprises:

detecting a distance between the mobile vehicle and the target object by using a distance measuring device when the target object corresponding to the image detection object is detected; and

performing the step (d) if the distance between the mobile vehicle and the target object is less than a threshold.

6. The driving notification method as claimed in claim 1 , wherein the step (d) comprises:

determining whether a distance between the target position of the image detection object on the surrounding image and the gaze position of the driver is greater than a threshold,

wherein the gaze position does not match the target position if the distance between the target position of the image detection object on the surrounding image and the gaze position of the driver is greater than the threshold; and the gaze position matches the target position if the distance between the target position of the image detection object on the surrounding image and the gaze position of the driver is less than or equal to the threshold.

7. A driving notification system disposed on a mobile vehicle, comprising:

an image capturing device;

an eye tracking device;

a memory, storing a plurality of instructions; and

a processor, coupled to the image capturing device, the eye tracking device, and the memory, wherein the processor is configured to execute the instructions to:

capture a surrounding image through the image capturing device;

perform a target object detection to the surrounding image to obtain a target position of an image detection object;

detect a gaze position of a driver through the eye tracking device;

determine whether the gaze position matches the target position; and

control an alarm device to activate an alarming operation if the gaze position does not match the target position,

wherein the processor is further configured to execute the instructions to:

obtain a reference set value of a projection plane of the eye tracking device and a projection position reflecting an eye gaze of the driver on the projection plane, wherein the reference set value comprises a width and a height of the projection plane, and a distance between the projection plane and the mobile vehicle; and

convert the projection position on the projection plane into a relative position on the surrounding image to obtain the gaze position of the driver according to a field of view of the image capturing device and the reference set value of the projection plane,

wherein the processor is further configured to execute the instructions to:

obtain a specific block on the surrounding image by serving the gaze position of the driver as an extending basis, wherein a size of the specific block corresponds to a size of the image detection object; and

determine image similarity between the specific block and the image detection object,

determine that the gaze position matches the target position if the image similarity between the specific block and the image detection object is greater than a threshold, and

determine that the gaze position does not match the target position if the image similarity between the specific block and the image detection object is less than or equal to the threshold.

8. The driving notification system as claimed in claim 7 , wherein the processor is further configured to execute the instructions to:

control the alarm device not to activate the alarming operation if the gaze position matches the target position.

9. The driving notification system as claimed in claim 7 , wherein the processor is coupled to a vehicle driving element of the mobile vehicle, and the processor is further configured to execute the instructions to:

control the alarm device to stop performing the alarming operation if the gaze position matches the target position; and

control the vehicle driving element of the mobile vehicle according to a target object corresponding to the image detection object if the gaze position does not match the target position.

10. The driving notification system as claimed in claim 9 , wherein the target object comprises a traffic sign or an obstruction, and the processor is further configured to execute the instructions to:

control a speed control device of the mobile vehicle according to the traffic sign or the obstruction to lower a driving speed of the mobile vehicle.

11. The driving notification system as claimed in claim 7 , wherein the driving notification system further comprises a distance measuring device coupled to the processor, and the processor is further configured to execute the instructions to:

detect a distance between the mobile vehicle and the target object by using the distance measuring device when the target object corresponding to the image detection object is detected; and

determine whether the gaze position matches the target position if the distance between the mobile vehicle and the target object is less than a threshold.

12. The driving notification system as claimed in claim 7 , wherein the processor is further configured to execute the instructions to:

determine whether a distance between the target position of the image detection object on the surrounding image and the gaze position of the driver is greater than a threshold,

wherein the gaze position does not match the target position if the distance between the target position of the image detection object on the surrounding image and the gaze position of the driver is greater than the threshold; and the gaze position matches the target position if the distance between the target position of the image detection object on the surrounding image and the gaze position of the driver is less than or equal to the threshold.

Assignments (3)
PATENT SECURITY AGREEMENT Recorded Mar 18, 2025
From: RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 070551/0860 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 6, 2025
From: ACER, INC
To: RPX CORPORATION
Reel/Frame 070422/0667 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2018
From: HO, ANDY; YANG, TSUNG-HAN; WANG, SZU-CHIEH; LIN, JIAN-CHI; HSIAO, JASON
To: ACER INCORPORATED
Reel/Frame 045215/0556 →