IP Library › Granted Patent US 9,932,000
Granted Patent B2
US 9,932,000 · App. 15/384,657 · Granted Apr 3, 2018

Information notification apparatus and information notification method

Inventors: Yasuhiro Aoki (Kawasaki, JP); Masami Mizutani (Mountain View, CA)
Assignee: FUJITSU LIMITED
B60R16/0231G06K9/00604G06K9/00617G06K9/00845H04N7/183
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,932,000
App. No.
15/384,657
Granted
Apr 3, 2018
Kind
B2
Abstract

An information notification apparatus includes a camera; a memory; and a processor coupled to the memory and configured to perform line-of-sight detection on an image captured by the camera, calculate a forward gaze degree of a gaze directed to a predetermined area in accordance with a line-of-sight direction obtained in line-of-sight detection; execute a notification to an outside in a first information if the forward gaze degree is lower than a first threshold; and execute the notification in a second information which information is less than the first information if the forward gaze degree is higher than or equal to the first threshold.

Claims (43)

1. An information notification apparatus comprising:

a camera configured to capture an image of a driver of a vehicle;

a memory; and

a processor coupled to the memory and configured to:

perform line-of-sight detection based on the image captured by the camera;

calculate a forward gaze degree of a gaze of the driver in accordance with a line-of-sight direction obtained in the line-of-sight detection, the forward gaze degree indicates whether the gaze of the driver is within a predetermined area of a direction in which the vehicle is traveling;

provide a notification to the driver in a first information when the forward gaze degree is lower than a first threshold; and

provide the notification to the driver in a second information when the forward gaze degree is higher than or equal to the first threshold, the second information includes less than the first information,

wherein the processor further executes the notification in the second information when the forward gaze degree is lower than a second threshold which is lower than the first threshold,

wherein the processor further classifies the forward gaze degree as a first group when a position at a time when the forward gaze degree was calculated is a traffic intersection, as a second group when an inter-vehicle distance at a time when the forward gaze degree was calculated is smaller than a predetermined value, and as a third group when the forward gaze degree is not classified as either the first group or the second group, after which the processor creates a forward gaze degree histogram for each of the first group, the second group, and the third group and sets the first threshold and the second threshold according to three forward gaze degree histograms related to the first group, the second group, and the third group.

2. The information notification apparatus according to claim 1 , further comprising a display,

wherein the notification is a display output directed to the display.

3. The information notification apparatus according to claim 2 , wherein:

according to a result in the line-of-sight detection, the processor further decides whether the notification that had been output as the display output in the second information has been visually recognized; and

when the notification that had been output as the display output in the second information has not been visually recognized within a predetermined time, the processor switches the display output of the notification from the second information to the first information.

4. The information notification apparatus according to claim 1 , wherein when a number of times at which the forward gaze degree higher than or equal to the first threshold is calculated is higher than or equal to a predetermined value, the processor further executes a circumference check notification that prompts the driver to check of a circumference of the vehicle.

5. The information notification apparatus according to claim 1 , wherein when a number of times which the forward gaze degree lower than the second threshold is calculated is higher than or equal to a predetermined value, the processor further executes a warning notification that warns the driver of being inattentive.

6. The information notification apparatus according to claim 1 , wherein the processor further sets a class value of the forward gaze degree to the second threshold, the class value being obtained at an intersection between a distribution curve of the forward gaze degree histogram related to the first group and a distribution curve of the forward gaze degree histogram related to the third group, and also sets a class value of the forward gaze degree to the first threshold, the class value being obtained at an intersection between a distribution curve of the forward gaze degree histogram related to the second group and a distribution curve of the forward gaze degree histogram related to the third group.

7. The information notification apparatus according to claim 1 , wherein the processor is further configured to:

classify images obtained by the camera based on the calculated forward gaze degree for each image, each image being classified as a forward visually recognized image or not a forward visually recognized image;

determine a ratio of a number of the classified forward visually recognized images to a number of images obtained by the camera within a predetermined period; and

provide one of the notification that prompts the driver to check of a circumference of the vehicle based on the determined ratio.

8. An information notification method comprising:

capturing, with a camera, an image of a driver of a vehicle;

performing, with a processor, line-of-sight detection based on the image captured by the camera,

calculating, with the processor, a forward gaze degree of a gaze of the driver in accordance with a line-of-sight direction obtained in line-of-sight detection, the forward gaze degree indicates whether the gaze of the driver is within a predetermined area of a direction in which the vehicle is travelling;

providing a notification to the driver in a first information when the forward gaze degree is lower than a first threshold; and

providing the notification in a second information when the forward gaze degree is higher than or equal to the first threshold, the second information includes less information than the first information,

wherein the processor further executes the notification in the second information when the forward gaze degree is lower than a second threshold which is lower than the first threshold,

wherein the processor further classifies the forward gaze degree as a first group when a position at a time when the forward gaze degree was calculated is a traffic intersection, as a second group when an inter-vehicle distance at a time when the forward gaze degree was calculated is smaller than a predetermined value, and as a third group when the forward gaze degree is not classified as either the first group or the second group, after which the processor creates a forward gaze degree histogram for each of the first group, the second group, and the third group and sets the first threshold and the second threshold according to three forward gaze degree histograms related to the first group, the second group, and the third group.

9. A vehicle-mounted apparatus to provide notifications to a driver, the vehicle-mounted apparatus comprising:

a notification device configured to present notifications to the driver of a vehicle in a plurality of forms based on a workload of the driver;

a memory;

a processor coupled to the memory and the at least one notification device, the processor configured to:

acquire a travelling direction of the vehicle;

perform line-of-sight detection to determine a line-of-sight for the driver;

calculate a gaze degree associated with the driver, the gaze degree indicating frequency of the determined line-of-sight of the driver included within a predetermined range of the travelling direction of the vehicle;

classify the workload of the driver as low when the calculated forward gaze degree is lower than a threshold;

classify the workload of the driver as high when the calculated forward gaze degree is higher than the threshold;

provide a notification to the at least one notification device in a first information when the workload of the driver is classified as low; and

provide the notification to the at least one notification device in a second information when the workload of the driver is classified as high, the second information includes less information than the first information,

wherein the line-of-sight detection determines a line-of-sight vector for the driver that interconnects a central position of a pupil of the eyeball and a center of the curvature of the cornea, and the processor calculates the gaze degree by determining whether an angle formed by the line-of-sight vector and a vector of the traveling direction of the vehicle is within a predetermined range,

wherein the processor further classifies the forward gaze degree as a first group when a position at a time when the forward gaze degree was calculated is a traffic intersection, as a second group when an inter-vehicle distance at a time when the forward gaze degree was calculated is smaller than a predetermined value, and as a third group when the forward gaze degree is not classified as either the first group or the second group, after which the processor creates a forward gaze degree histogram for each of the first group, the second group, and the third group and sets the first threshold and the second threshold according to three forward gaze degree histograms related to the first group, the second group, and the third group.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2016
From: AOKI, YASUHIRO; MIZUTANI, MASAMI
To: FUJITSU LIMITED
Reel/Frame 041034/0752 →
Priority Claims (1)
JP 2016-001349 · Jan 6, 2016 · national
Continuity (1)
Related Publication 20170190306A1 · Jul 6, 2017