IP Library › Granted Patent US 12,202,489
Granted Patent B2
US 12,202,489 · App. 18/343,144 · Granted Jan 21, 2025

Information processing device, moving apparatus, method, and program

Inventor: Eiji Oba (Tokyo, JP)
Assignee: Sony Semiconductor Solutions Corporation
B60W40/08B60W50/082B60W50/14G06V20/597B60W2040/0818B60W2040/0881B60W2050/007B60W2050/146B60W2540/223B60W2540/229
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,202,489
App. No.
18/343,144
Granted
Jan 21, 2025
Kind
B2
Abstract

An alertness level that is the consciousness level of the driver of a vehicle is determined, and the timing of a manual driving return request notification is controlled in accordance with the alertness level. A data processing unit determines the alertness level of the driver of the vehicle, and records evaluation data. The data processing unit acquires observation information about the driver, determines the alertness level of the driver, and evaluates and records the return rate at each time. The vehicle is a vehicle capable of switching between automatic driving and manual driving, and the data processing unit controls the timing of a manual driving return request notification for the driver's return from automatic driving to manual driving, in accordance with the alertness level of the driver. The observation information to be acquired by the data processing unit includes observation information about the driver both before and after the driver gets in the vehicle. For example, passive monitoring, information about the driver's response to a notification, and the like are observed.

Claims (33)

1. An information processing device comprising:

a processor configured to:

acquire observation information about a driver of a vehicle capable of switching between automatic driving and manual driving;

determine an alertness level of the driver based on the observation information;

determine a manual driving start time of the vehicle to return from automatic driving to manual driving; and

in response to the determined alertness level of the driver, control a notification timing to return from automatic driving to manual driving,

wherein, in response to the alertness level being below a threshold level, the notification timing is set to a first time earlier than the manual driving start time, and in response to the alertness level being above the threshold level, the notification timing is set to a second time earlier than the manual driving start time, the first time being earlier than the second time and wherein the notification timing is determined, by machine learning based on driver information and environment information, to obtain a certain safety index value or greater, the safety index value being indicative of a degree of safe driving.

2. The information processing device according to claim 1 , wherein a time between the first time and the manual driving start time is increased as the determined alertness level of the driver decreases.

3. The information processing device according to claim 1 , wherein the alertness level of the driver is determined in accordance with one of a plurality of driver states.

4. The information processing device according to claim 1 , wherein the observation information about the driver is acquired by passive monitoring of the driver.

5. The information processing device according to claim 1 , wherein the observation information about the driver is acquired by active monitoring of the driver.

6. The information processing device according to claim 1 , wherein the observation information about the driver is acquired from one or more images of the driver.

7. The information processing device according to claim 1 , wherein the observation information about the driver is based on eye movement of the driver.

8. The information processing device according to claim 1 , wherein the observation information about the driver indicates that the driver is away from a driver's seat.

9. An information processing method executed by a processor, the method comprising:

acquiring observation information about a driver of a vehicle capable of switching between automatic driving and manual driving;

determining an alertness level of the driver based on the observation information;

determining a manual driving start time of the vehicle to return from automatic driving to manual driving; and

in response to the determined alertness level of the driver, controlling a notification timing to return from automatic driving to manual driving,

wherein, in response to the alertness level being below a threshold level, the notification timing is set to a first time earlier than the manual driving start time, and in response to the alertness level being above the threshold level, the notification timing is set to a second time earlier than the manual driving start time, the first time being earlier than the second time and wherein the notification timing is determined, by machine learning based on driver information and environment information, to obtain a certain safety index value or greater, the safety index value being indicative of a degree of safe driving.

10. The information processing method according to claim 9 , wherein a time between the first time and the manual driving start time is increased as the determined alertness level of the driver decreases.

11. The information processing method according to claim 9 , wherein the alertness level of the driver is determined in accordance with one of a plurality of driver states.

12. The information processing method according to claim 9 , wherein the observation information about the driver is acquired by passive monitoring of the driver.

13. The information processing method according to claim 9 , wherein the observation information about the driver is acquired by active monitoring of the driver.

14. The information processing method according to claim 9 , wherein the observation information about the driver is acquired from one or more images of the driver.

15. The information processing method according to claim 9 , wherein the observation information about the driver is based on eye movement of the driver.

16. The information processing method according to claim 9 , wherein the observation information about the driver indicates that the driver is away from a driver's seat.

17. A non-transitory computer readable medium containing instructions that, when executed by a processor, perform an information processing method comprising:

acquiring observation information about a driver of a vehicle capable of switching between automatic driving and manual driving;

determining an alertness level of the driver based on the observation information;

determining a manual driving start time of the vehicle to return from automatic driving to manual driving; and

in response to the determined alertness level of the driver, controlling a notification timing to return from automatic driving to manual driving,

wherein, in response to the alertness level being below a threshold level, the notification timing is set to a first time earlier than the manual driving start time, and in response to the alertness level being above the threshold level, the notification timing is set to a second time earlier than the manual driving start time, the first time being earlier than the second time and wherein the notification timing is determined, by machine learning based on driver information and environment information, to obtain a certain safety index value or greater, the safety index value being indicative of a degree of safe driving.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 27, 2023
From: OBA, EIJI
To: SONY SEMICONDUCTOR SOLUTIONS CORPORATION
Reel/Frame 065047/0936 →
Priority Claims (1)
JP 2018-171586 · Sep 13, 2018 · national
Continuity (3)
Continuation 17888200 · Aug 15, 2022
Continuation 17274100
Related Publication 20230339476A1 · Oct 26, 2023
References Cited (27)
US 11420636B2 · Oba · 2022 [cited by applicant]
US 20160029940A1 · Iizuka et al. · 2016 [cited by applicant]
US 20160041553A1 · Sato et al. · 2016 [cited by applicant]
US 20170303842A1 · Yoshida et al. · 2017 [cited by applicant]
US 20180113454A1 · Emura et al. · 2018 [cited by applicant]
US 20180329414A1 · Igarashi · 2018 [cited by examiner]
US 20190187701A1 · Zheng · 2019 [cited by examiner]
US 20190339695A1 · Feron · 2019 [cited by examiner]
US 20200064833A1 · Fox · 2020 [cited by examiner]
US 20210188289A1 · Oba · 2021 [cited by applicant]
US 20220388518A1 · Oba · 2022 [cited by applicant]
CN 103476658A · 2013 [cited by applicant]
CN 105072986A · 2015 [cited by applicant]
CN 105365823A · 2016 [cited by applicant]
CN 107635844A · 2018 [cited by applicant]
EP 2977975A1 · 2016 [cited by applicant]
EP 2982565A2 · 2016 [cited by applicant]
EP 3378722A1 · 2018 [cited by applicant]
JP 2016038768A · 2016 [cited by applicant]
JP 2017001563A · 2017 [cited by applicant]
JP 2017131445A · 2017 [cited by applicant]
JP 2018045451A · 2018 [cited by applicant]
WO WO2014147828A1 · 2014 [cited by applicant]
WO WO2017085981A1 · 2017 [cited by applicant]
International Search Report and English translation thereof mailed Nov. 26, 2019 in connection with International Application No. PCT/JP2019/034078. [cited by applicant]
Written Opinion and English translation thereof mailed Nov. 26, 2019 in connection with International Application No. PCT/JP2019/034078. [cited by applicant]
International Preliminary Report on Patentability and English translation thereof mailed Mar. 25, 2021 in connection with International Application No. PCT/JP2019/034078. [cited by applicant]