IP Library › Granted Patent US 11,427,216
Granted Patent B2
US 11,427,216 · App. 16/433,867 · Granted Aug 30, 2022

User activity-based customization of vehicle prompts

Inventors: Michal Genussov Ziss (Tel-Aviv, IL); Kobi Nistel (Kadima-Zoran, IL); Gila Kamhi (Zichron Yaakov, IL); Eli Tzirkel-Hancock (Ra'anana, IL)
Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
B60W50/085G06V20/597G10L13/00B60W2050/0082
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,427,216
App. No.
16/433,867
Granted
Aug 30, 2022
Kind
B2
Abstract

A system for interacting with a user in a vehicle includes a processing device including an input module configured to receive measurement data from one or more sensors, the measurement data related to features of the user. The system also includes a prompt design module configured to, in response to receiving an instruction to output a requested prompt to the user, determine a user state based on the measurement data, determine an activity state based on the measurement data and the user state, customize the requested prompt based on the user state and the activity state to generate a customized prompt, and output the customized prompt to the user.

Claims (38)

1. A system for interacting with a user in a vehicle, comprising:

a processing device including an input module having a first processor configured to receive measurement data from one or more sensors, the measurement data related to features of the user;

a prompt design module having a second processor configured to, in response to receiving an instruction to output a requested prompt to the user, perform:

determining a user state based on the measurement data;

determining an activity state based on the measurement data and the user state;

determining a level of urgency associated with the requested prompt;

identifying a base prompt based on the requested prompt, the base prompt stored in a data structure, the data structure including a plurality of stored base prompts, each stored base prompt related in the data structure to one or more stored user states and one or more stored activity states, each stored base prompt related to a plurality of stored customized prompts configured to present a message to the user, the plurality of stored customized prompts including at least one full format prompt having a full version of the message and at least one short format prompt having a short version of the message;

customizing the identified base prompt based on the user state and the activity state to generate a customized prompt, wherein customizing includes selecting a stored user state related to the identified base prompt based on the determined user state, selecting a stored activity state related to the selected stored user state, and selecting a stored customized prompt related to the selected stored activity state, wherein the selected stored customized prompt is a full format prompt based on the level of urgency being lower than a reference level, and the selected stored customized prompt is a short format prompt based on the level of urgency being equal to or higher than the reference level; and

outputting the selected stored customized prompt to the user.

2. The system of claim 1 , wherein the customized prompt is a speech prompt generated by a speech generation module.

3. The system of claim 1 , wherein each stored activity state is a sub-state of a related user state in the data structure.

4. The system of claim 3 , wherein the user state is selected from a plurality of stored user states, and each stored user state of the plurality of stored user states is related to one or more features estimated based on the measurement data.

5. The system of claim 4 , wherein each stored user state is related to a plurality of stored activity states, and each stored activity state of the plurality of stored activity states is related to a stored customized prompt.

6. The system of claim 5 , wherein the stored activity state is selected from a plurality of stored activity states related to the user state.

7. The system of claim 5 , wherein the user is one of a plurality of users, and the plurality of stored user states, the one or more stored activity states and the stored customized prompt are stored in a data structure specific to the user.

8. The system of claim 7 , wherein the data structure is a look-up table.

9. The system of claim 7 , wherein the stored customized prompt based on a preference of the user.

10. The system of claim 1 , wherein the prompt design module is configured to further customize the requested prompt based on

contextual information related to at least one of an interior of the vehicle and a condition external to the vehicle.

11. A method of interacting with a user in a vehicle, comprising:

receiving measurement data from one or more sensors, the measurement data related to features of the user;

in response to receiving an instruction to output a requested prompt to the user, performing:

determining a user state based on the measurement data;

determining an activity state based on the measurement data and the user state;

determining a level of urgency associated with the requested prompt;

identifying a base prompt having an initial format based on the requested prompt, the base prompt stored in a data structure, the data structure including a plurality of stored base prompts, each stored base prompt related in the data structure to one or more stored user states and one or more stored activity states each stored base prompt related to a plurality of stored customized prompts configured to present a message to the user, the plurality of stored customized prompts including at least one full format prompt having a full version of the message and at least one short format prompt having a short version of the message;

customizing the identified base prompt based on the user state and the activity state to generate a customized prompt, wherein customizing includes selecting a stored user state related to the identified base prompt based on the determined user state, selecting a stored activity state related to the selected stored user state, and selecting a stored customized prompt related to the selected stored activity state, wherein the selected stored customized prompt is a full format prompt based on the level of urgency being lower than a reference level, and the selected stored customized prompt is a short format prompt based on the level of urgency being equal to or higher than the reference level; and

outputting the selected stored customized prompt to the user.

12. The method of claim 11 , wherein the customized prompt is a speech prompt generated by a speech generation module.

13. The method of claim 11 , wherein each stored activity state is a sub-state of a related user state in the data structure.

14. The method of claim 13 , wherein the user state is selected from a plurality of stored user states, and each stored user state of the plurality of stored user states is related to one or more features estimated based on the measurement data.

15. The method of claim 14 , wherein each stored user state is related to a plurality of stored activity states, and each stored activity state of the plurality of stored activity states is related to a stored customized prompt.

16. The method of claim 15 , wherein the stored activity state is selected from a plurality of stored activity states related to the user state.

17. The method of claim 15 , wherein the user is one of a plurality of users, and the plurality of stored user states, the one or more stored activity states and the stored customized prompt are stored in a data structure specific to the user.

18. The method of claim 17 , wherein the data structure is a look-up table.

19. The method of claim 17 , wherein the stored customized prompt is based on a preference of the user.

20. The method of claim 11 , further comprising customizing the requested prompt based on

contextual information related to at least one of an interior of the vehicle and a condition external to the vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2019
From: GENUSSOV ZISS, MICHAL; NISTEL, KOBI; KAMHI, GILA; TZIRKEL-HANCOCK, ELI
To: GM GLOBAL TECHNOLOGY OPERATIONS LLC
Reel/Frame 049397/0711 →
Continuity (1)
Related Publication 20200385011A1 · Dec 10, 2020