IP Library Granted Patent US 12,017,676
Granted Patent B2
US 12,017,676 · App. 18/072,352 · Granted Jun 25, 2024

Driver monitoring device, driver monitoring method, and non-transitory computer-readable medium

Inventors: Ryosuke Muratani (Kanagawa, JP); Satoshi Shimizu (Kanagawa, JP); Shinya Ema (Kanagawa, JP); Kunio Fujikawa (Kanagawa, JP); Kento Mizuta (Kanagawa, JP); Yucheng(Leo) Zhang (Kanagawa, JP); Kisei Takada (Kanagawa, JP)
Assignee: PANASONIC AUTOMOTIVE SYSTEMS CO., LTD.
B60W50/14B60W40/08G06V20/597G06V40/103G06V40/174G06V40/18B60W2040/0827B60W2050/143B60W2050/146B60W2420/403B60W2540/21B60W2540/223B60W2540/229
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,017,676
App. No.
18/072,352
Granted
Jun 25, 2024
Kind
B2
Abstract

A driver monitoring device includes: a processor; and a memory. The memory stores instructions, when executed by the processor, cause the driver monitoring device to perform operations including: detecting an unbalanced posture of a driver based on an image of a driver's seat; determining whether the unbalanced posture is a habitual unbalanced posture of the driver; calculating an opening degree of at least one eye of the driver based on the image in a case in which it is determined that the unbalanced posture is other than the habitual unbalanced posture; determining whether it is a timing to give a notification in a case in which the vehicle is traveling and the opening degree is equal to or larger than a first threshold; and giving the notification to the driver in a case in which it is determined that it is the timing to give the notification.

Claims (55)

1. A driver monitoring device comprising:

a processor; and

a memory storing instructions that, when executed by the processor, cause the driver monitoring device to perform operations, the operations comprising:

detecting an unbalanced posture of a driver based on an image of a driver's seat of a vehicle captured by a camera;

determining whether the unbalanced posture of the driver is a habitual unbalanced posture of the driver;

calculating an opening degree of at least one eye of the driver based on the image in a case in which it is determined that the unbalanced posture is other than the habitual unbalanced posture of the driver;

determining whether it is a timing to give a notification to the driver in a case in which the vehicle is traveling and the opening degree is equal to or larger than a first threshold; and

giving the notification to the driver in a case in which it is determined that it is the timing to give the notification.

2. The driver monitoring device according to claim 1 ,

wherein the operations further comprise:

determining that it is the timing to give the notification in a case in which a distance from a current position of the vehicle to a rest place present on a route on which the vehicle is scheduled to travel is less than a second threshold; and

giving, as the notification, a first notification for urging the driver to take a rest at the rest place.

3. The driver monitoring device according to claim 1 ,

wherein the operations further comprise:

determining that it is the timing to give the notification in a case in which a distance from the current position of the vehicle to a place where attention needs to be paid to traveling present on a route on which the vehicle is scheduled to travel is less than a third threshold; and

giving, as the predetermined notification, a second notification for urging the driver to perform safe driving.

4. The driver monitoring device according to claim 3 ,

wherein the place where attention needs to be paid to traveling includes at least one of a frequent accident occurrence point, an accident occurrence point, and a point under construction.

5. The driver monitoring device according to claim 1 ,

wherein the operations further comprise:

determining that it is the timing to give the notification in a case in which it is determined that the driver is in a conversation as a result of analysis of a facial expression of the driver included in the image; and

giving, as the notification, a third notification for urging the driver to concentrate on driving.

6. The driver monitoring device according to claim 1 , further comprising:

determining that it is the timing to give the notification in a case in which it is determined that the driver is in a conversation as a result of analysis of a voice collected by a microphone in the vehicle; and

giving, as the notification, a third notification for urging the driver to concentrate on driving.

7. The driver monitoring device according to claim 1 ,

wherein the determining whether it is a timing to give a notification to the driver is performed further in a case in which the vehicle is traveling and a state in which the opening degree is smaller than the first threshold continues for a period equal to or shorter than a fourth threshold.

8. A driver monitoring method comprising:

detecting an unbalanced posture of a driver based on an image of a driver's seat of a vehicle captured by a camera;

determining whether the unbalanced posture of the driver is a habitual unbalanced posture of the driver;

calculating an opening degree of at least one eye of the driver based on the image in a case in which it is determined that the unbalanced posture is other than the habitual unbalanced posture of the driver;

determining whether it is a timing to give a notification to the driver in a case in which the vehicle is traveling and the opening degree is equal to or larger than a first threshold; and

giving the notification to the driver in a case in which it is determined that it is the timing to give the notification.

9. The driver monitoring method according to claim 8 , further comprising:

determining that it is the timing to give the notification in a case in which a distance from a current position of the vehicle to a rest place present on a route on which the vehicle is scheduled to travel is less than a second threshold; and

giving, as the notification, a first notification for urging the driver to take a rest at the rest place.

10. The driver monitoring method according to claim 8 , further comprising:

determining that it is the timing to give the notification in a case in which a distance from the current position of the vehicle to a place where attention needs to be paid to traveling present on a route on which the vehicle is scheduled to travel is less than a third threshold; and

giving, as the predetermined notification, a second notification for urging the driver to perform safe driving.

11. The driver monitoring method according to claim 10 ,

wherein the place where attention needs to be paid to traveling includes at least one of a frequent accident occurrence point, an accident occurrence point, and a point under construction.

12. The driver monitoring method according to claim 8 , further comprising:

determining that it is the timing to give the notification in a case in which it is determined that the driver is in a conversation as a result of analysis of a facial expression of the driver included in the image; and

giving, as the notification, a third notification for urging the driver to concentrate on driving.

13. The driver monitoring method according to claim 8 , further comprising:

determining that it is the timing to give the notification in a case in which it is determined that the driver is in a conversation as a result of analysis of a voice collected by a microphone in the vehicle; and

giving, as the notification, a third notification for urging the driver to concentrate on driving.

14. The driver monitoring method according to claim 8 ,

wherein the determining whether it is a timing to give a notification to the driver is performed further in a case in which the vehicle is traveling and a state in which the opening degree is smaller than the first threshold continues for a period equal to or shorter than a fourth threshold.

15. A non-transitory computer-readable medium that stores a driver monitoring program, the driver monitoring program, when executed by a processor, causing a computer to perform a process, the process comprising:

detecting an unbalanced posture of a driver based on an image of a driver's seat of a vehicle captured by a camera;

determining whether the unbalanced posture of the driver is a habitual unbalanced posture of the driver;

calculating an opening degree of at least one eye of the driver based on the image in a case in which it is determined that the unbalanced posture is other than the habitual unbalanced posture of the driver;

determining whether it is a timing to give a notification to the driver in a case in which the vehicle is traveling and the opening degree is equal to or larger than a first threshold; and

giving the notification to the driver in a case in which it is determined that it is the timing to give the notification.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2024
From: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
To: PANASONIC AUTOMOTIVE SYSTEMS CO., LTD.
Reel/Frame 066709/0745 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 27, 2023
From: MURATANI, RYOSUKE; SHIMIZU, SATOSHI; EMA, SHINYA; FUJIKAWA, KUNIO; MIZUTA, KENTO; ZHANG, YUCHENG(LEO); TAKADA, KISEI
To: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
Reel/Frame 063104/0386 →
Priority Claims (1)
JP 2021-200980 · Dec 10, 2021 · national
Continuity (1)
Related Publication 20230182762A1 · Jun 15, 2023