IP Library Granted Patent US 11,749,032
Granted Patent B2
US 11,749,032 · App. 17/321,939 · Granted Sep 5, 2023

Systems and methods for adapting notifications according to component monitoring

Inventors: Hiroshi Yasuda (San Francisco, CA); Manuel Ludwig Kuehner (Mountain View, CA)
Assignee: Toyota Research Institute, Inc.
G07C5/0825B60Q9/00G06V20/597G06V40/19G08B5/36G08B21/24G08B21/06
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,749,032
App. No.
17/321,939
Granted
Sep 5, 2023
Kind
B2
Abstract

System, methods, and other embodiments described herein relate to adapting notifications according to monitoring states of a vehicle operator. In one embodiment, a method includes acquiring sensor data associated with an operator of a vehicle. The method also includes determining an operator state from the sensor data. The method also includes computing a difference of the operator state to a parameter associated with monitoring a component in the vehicle. The method also includes adapting a notification associated with the component by a controller according to the difference.

Claims (38)

1. A notification system comprising:

a memory communicably coupled to a processor and storing:

a notification module including instructions that when executed by the processor cause the processor to:

acquire sensor data associated with an operator of a vehicle using automated driving for a maneuver;

determine an operator state from the sensor data using a derived body angle and a geometric profile of the operator towards a component and another component associated with the vehicle, the operator state indicating excessive monitoring of the another component for the maneuver and an urgency level;

compute a difference of the operator state to a parameter associated with monitoring the component in the vehicle for the maneuver; and

adapt a notification associated with the component by a controller according to insufficient awareness estimated from the difference, and the notification factors training the operator for using the component during the maneuver.

2. The notification system of claim 1 , wherein the notification module includes instructions to determine the operator state further including instructions to monitor a gaze of the operator to the component and the another component according to the sensor data and compute an average of the operator state according to a frequency of the gaze, the body angle, and the geometric profile.

3. The notification system of claim 2 , wherein the notification module includes instructions to compute the difference further including instructions to compare the average to the parameter in association with adapting the notification, wherein the parameter is different for the maneuver during a manual mode.

4. The notification system of claim 1 , wherein the notification module includes instructions to compute the difference further including instructions to compare the insufficient awareness of the operator according to the sensor data to the parameter and wherein instructions to adapt the notification further include instructions to change illumination of the component and an operation of the another component in relation to minimizing the difference.

5. The notification system of claim 1 , wherein the notification module further includes instructions to control the component to change illumination in association with the notification.

6. The notification system of claim 1 , wherein the notification module further includes instructions to compute a moving average of the operator state that increases or decreases according to a gaze frequency by the operator to the component and an operation of the another component, and to control illumination of the component by the controller according to the moving average changing and the parameter, wherein a window size of the moving average is a cost function for the operator state that factors a complexity associated with the maneuver.

7. The notification system of claim 6 , wherein the moving average is proportional to the gaze frequency and a vehicle state.

8. The notification system of claim 1 , wherein the parameter is set according to conditions related to the automated driving or illumination of the component for an environmental state of the vehicle.

9. A non-transitory computer-readable medium comprising:

instructions that when executed by a processor cause the processor to:

acquire sensor data associated with an operator of a vehicle using automated driving for a maneuver;

determine an operator state from the sensor data using a derived body angle and a geometric profile of the operator towards a component and another component associated with the vehicle, the operator state indicating excessive monitoring of the another component for the maneuver and an urgency level;

compute a difference of the operator state to a parameter associated with monitoring the component in the vehicle for the maneuver; and

adapt a notification associated with the component by a controller according to insufficient awareness estimated from the difference, and the notification factors training the operator for using the component during the maneuver.

10. The non-transitory computer-readable medium of claim 9 , wherein the instructions to determine the operator state further include instructions to monitor a gaze of the operator to the component and the another component according to the sensor data and compute an average of the operator state according to a frequency of the gaze, the body angle, and the geometric profile.

11. The non-transitory computer-readable medium of claim 10 , wherein the instructions to compute the difference further include instructions to compare the average to the parameter in association with adapting the notification, wherein the parameter is different for the maneuver during a manual mode.

12. The non-transitory computer-readable medium of claim 9 , wherein the instructions to compute the difference further include instructions to compare the insufficient awareness of the operator according to the sensor data to the parameter and wherein the instructions to adapt the notification further include instructions to change illumination of the component and an operation of the another component in relation to minimizing the difference.

13. A method, comprising:

acquiring sensor data associated with an operator of a vehicle using automated driving for a maneuver;

determining an operator state from the sensor data using a derived body angle and a geometric profile of the operator towards a component and another component associated with the vehicle, the operator state indicating excessive monitoring of the another component for the maneuver and an urgency level;

computing a difference of the operator state to a parameter associated with monitoring the component in the vehicle for the maneuver; and

adapting a notification associated with the component by a controller according to insufficient awareness estimated from the difference, and the notification factors training the operator for using the component during the maneuver.

14. The method of claim 13 , wherein determining the operator state further includes monitoring a gaze of the operator to the component and the another component according to the sensor data and computing an average of the operator state according to a frequency of the gaze, the body angle, and the geometric profile.

15. The method of claim 14 , wherein computing the difference further includes comparing the average to the parameter in association with adapting the notification wherein the parameter is different for the maneuver during a manual mode.

16. The method of claim 13 , wherein computing the difference further includes comparing the insufficient awareness of the operator according to the sensor data to the parameter and wherein adapting the notification further includes changing illumination of the component and an operation of the another component in relation to minimizing the difference.

17. The method of claim 13 , further comprising:

controlling the component to change illumination in association with the notification.

18. The method of claim 13 , further comprising:

computing a moving average of the operator state that increases or decreases according to a gaze frequency by the operator to the component and an operation of the another component; and

controlling illumination of the component by the controller according to the moving average changing and the parameter, wherein a window size of the moving average is a cost function for the operator state that factors a complexity associated with the maneuver.

19. The method of claim 18 , wherein the moving average is proportional to the gaze frequency and a vehicle state.

20. The method of claim 13 , wherein the parameter is set according to conditions related to the automated driving or illumination of the component for an environmental state of the vehicle.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 18, 2023
From: TOYOTA RESEARCH INSTITUTE, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 064937/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 20, 2021
From: YASUDA, HIROSHI; KUEHNER, MANUEL LUDWIG
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 056297/0597 →
Continuity (1)
Related Publication 20220366737A1 · Nov 17, 2022