IP Library Granted Patent US 11,200,417
Granted Patent B2
US 11,200,417 · App. 16/962,383 · Granted Dec 14, 2021

Determination apparatus and method to enhance accuracy in determining the presence of an erroneously detected eyelid

Inventors: Kazuki Kunihiro (Tokyo, JP); Yudai Nakamura (Tokyo, JP); Takashi Hirano (Tokyo, JP)
Assignee: MITSUBISHI ELECTRIC CORPORATION
G06K9/0061G06K9/00604G06K9/00845G06K9/4661
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,200,417
App. No.
16/962,383
Granted
Dec 14, 2021
Kind
B2
Abstract

An area extraction unit ( 2 ) extracts an eye area in an image of a driver. An eyelid detection unit ( 3 ) detects an eyelid in the eye area. A reliability calculation unit ( 4 ) calculates an eyelid reliability by using luminance information of the eye area and positional information of the eyelid detected by the eyelid detection unit ( 3 ). A determination unit ( 5 ) determines that the eyelid has not been properly detected, when the eyelid reliability is less than a first threshold value.

Claims (39)

1. An erroneous detection determination apparatus, comprising:

processing circuitry to acquire image data indicating an image of a driver;

to extract an eye area in the image by using the image data acquired;

to detect an eyelid in the eye area extracted;

to calculate an eyelid reliability by using luminance information of the eye area extracted and positional information of the eyelid detected; and

to determine that the eyelid has not been properly detected, when the eyelid reliability calculated is less than a first threshold value,

wherein the processing circuitry calculates, for each of pixels corresponding to the eyelid detected, an inner product of a luminance vector obtained from the luminance information and a normal vector of an eyelid line obtained from the positional information, calculates a total value of the inner products calculated for the respective pixels. and calculates the eyelid reliability on a basis of the total value.

2. The erroneous detection determination apparatus of claim 1 , wherein the processing circuitry

calculates an eye open-closed state by using the positional information of the eyelid detected, and

when it is determined that the eyelid has not been properly detected, determines that the calculated open-closed state is invalid.

3. The erroneous detection determination apparatus of claim 1 , wherein

when, in images indicated by respective pieces of image data having been acquired by the processing circuitry, a ratio of images each with the eyelid reliability of ess than a second threshold value, is not less than a third threshold value,

the processing circuitry decreases the eyelid reliability that is calculated by using the luminance information and the positional information of the eyelid, and

performs the determination as to whether the eyelid has not been properly detected, by using the eyelid reliability decreased.

4. The erroneous detection determination apparatus of claim 2 , wherein

when, in images indicated by respective pieces of image data having been acquired by the processing circuitry, a ratio of images each with the eyelid reliability of less than a second threshold value, is not less than a third threshold value,

the processing circuitry decreases the eyelid reliability that is calculated by using the luminance information and the positional information of the eyelid, and

performs the determination as to whether the eyelid has not been properly detected, by using the eyelid reliability decreased.

5. An erroneous detection determination apparatus, comprising:

processing circuitry to acquire image data indicating an image of a driver;

to extract an eye area in the image by using the image data acquired;

to detect an eyelid in the eye area extracted;

to calculate an eyelid reliability by using luminance information of the eye area extracted and positional information of the eyelid detected;

to determine that the eyelid has not been properly detected, when the eyelid reliability calculated is less than a first threshold value; and

to calculate an eye open-closed state by using the positional information of the eyelid detected, wherein

when it is determined that the eyelid has not been properly detected, the processing circuitry determines that the calculated open-closed state is invalid, and

when, in images indicated by respective pieces of image data having been acquired by the processing circuitry, a ratio of images each with the eyelid reliability of less than a second threshold value, is not less than a third threshold value,

the processing circuitry decreases the eyelid reliability that is calculated by using the luminance information and the positional information of the eyelid, and

performs the determination as to whether the eyelid has not been properly detected, by using the eyelid reliability decreased.

6. The erroneous detection determination apparatus of claim 5 , wherein the processing circuitry calculates the eyelid reliability by using inner products of respective luminance vectors obtained from the luminance information and respective normal vectors of an eyelid line obtained from the positional information.

7. An erroneous detection determination method, comprising:

acquiring image data indicating an image of a driver;

extracting an eye area in the image by using the image data acquired;

detecting an eyelid in the eye area extracted;

calculating an eyelid reliability by using luminance information of the eye area extracted and positional information of the eyelid detected;

determining that the eyelid has not been properly detected, when the eyelid reliability calculated is less than a first threshold value;

calculating, for each of pixels corresponding to the eyelid detected, an inner product of a luminance vector obtained from the luminance information and a normal vector of an eyelid line obtained from the positional information; and

calculating a total value of the inner products calculated for the respective pixels,

wherein the eyelid reliability is calculated on a basis of the total value.

Assignments (2)
COMPANY SPLIT Recorded Sep 4, 2024
From: MITSUBISHI ELECTRIC CORPORATION
To: MITSUBISHI ELECTRIC MOBILITY CORPORATION
Reel/Frame 068834/0585 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 15, 2020
From: KUNIHIRO, KAZUKI; NAKAMURA, YUDAI; HIRANO, TAKASHI
To: MITSUBISHI ELECTRIC CORPORATION
Reel/Frame 053219/0414 →