IP Library › Granted Patent US 12,573,399
Granted Patent B2
US 12,573,399 · App. 18/565,573 · Granted Mar 10, 2026

Display control device and display control method

Inventors: Masaru Tasaki (Kanagawa, JP); Eiji Sonezaki (Kanagawa, JP)
Assignee: Nissan Motor Co., Ltd.
G10L15/22G06F3/167G10L15/01G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,573,399
App. No.
18/565,573
Filed
Nov 30, 2023
Granted
Mar 10, 2026
Kind
B2
Examiner
AZAD, ABUL K
Art Unit
2656
USPC
704/251
Abstract

A display control device includes a controller having a voice recognition unit and a display control unit. The voice recognition unit detects voice input from an occupant and recognizes the voice by voice processing of the detected voice. The display control unit causes a display to display a main screen including an operating state image indicating the operating state of a dialogue system, and a category screen including a category image in which commands are grouped. The controller causes the display to display the main screen before the voice recognition unit detects the voice, causes the display to display the category screen when the voice processing is not completed normally, and changes the display screen from the category screen to another screen when a category is selected in a state in which the category screen is displayed.

Claims (51)

1 . A display control device that controls a display screen of a display based on a voice of an occupant of a vehicle, the display control device comprising:

a controller including a voice recognition unit configured to detect a voice input of the occupant and recognize the voice by voice processing of a detected voice; and

a display control unit configured to control the display to display a main screen including an operating state image indicating an operating state of a dialogue system and to display a category screen including a category image in which commands are grouped,

the controller being configured to

control the display to display the main screen before the voice recognition unit detects the voice,

control the display to display the category screen when the voice processing is not completed normally in the main screen, and

change the display screen from the category screen to another screen when a category is selected in a state in which the category screen is displayed.

2 . The display control device according to claim 1 , wherein

the display control unit is configured to control the display to display a list screen including a list of the commands that have been grouped.

3 . The display control device according to claim 2 , wherein

the controller is configured to determine whether the list is selected based on any one of the voice of the occupant, a touch operation on the display by the occupant, or a switch operation by the occupant while the list screen is displayed on the display.

4 . The display control device according to claim 2 , wherein

the list screen includes sample text,

the list is a list grouping a plurality of syntaxes each having at least a command, and

the sample text includes information pertaining to either a character string or a numerical string assigned to the syntaxes.

5 . The display control device according to claim 4 , wherein

the controller is configured to control the display to display a screen for selecting sample text of either the character string or the numerical string assigned to the syntaxes.

6 . The display control device according to claim 2 , wherein

the list screen does not include animation.

7 . The display control device according to claim 1 , wherein

the operating state image includes an image of at least one of a message indicating the operating state of the dialogue system and an animation that moves in accordance with the operating state of the dialogue system, and

the category screen includes an image of at least one of a name of the category and an icon of the category.

8 . The display control device according to claim 1 , wherein

the category screen includes an animation that moves in accordance with the operating state of the dialogue system.

9 . The display control device according to claim 1 , wherein

the controller is configured to control the display to display the category screen when a prescribed period of time has elapsed in a state in which the voice input can be accepted before the voice processing is completed normally.

10 . The display control device according to claim 1 , wherein

the controller is configured to determine an abnormality of the voice processing when the command cannot be recognized from a content of an occupant's utterance, and to control the display to display the category screen upon determining that the voice processing is abnormal.

11 . The display control device according to claim 1 , wherein

the controller is configured to determine whether the category is selected based on any one of the voice of the occupant, a touch operation on the display by the occupant, or a switch operation by the occupant while the category screen is displayed on the display.

12 . The display control device according to claim 1 , wherein

the main screen includes a sample message including the commands.

13 . The display control device according to claim 1 , wherein

the controller being further configured to determine that the voice processing is not completed normally upon determining that the voice input of the occupant is not received within a predetermined amount of time after displaying the main screen or upon determining occurrence of an abnormality during the voice processing.

14 . A display control method for controlling a display screen of a display based on a voice of an occupant of a vehicle, the display control method comprising:

activating a dialogue system that interacts with the occupant to provide a service;

outputting a control command to the display to control the display to display a main screen including an operating state image indicating an operating state of the dialogue system while in a state before the dialogue system detects the voice;

detecting a voice input of the occupant and executing voice processing of a detected voice;

outputting a control command to the display to display on the main screen for displaying a category screen including a category image in which commands are grouped when the voice processing is not completed normally; and

outputting a control command to the display to change the category screen to another screen when a category is selected in a state in which the category screen is displayed.

15 . The display control method according to claim 14 , wherein

the voice processing is not completed normally upon determining that the voice input of the occupant is not received within a predetermined amount of time after displaying the main screen or upon determining occurrence of an abnormality during the voice processing.

16 . A display control device that controls a display screen of a display based on a voice of an occupant of a vehicle, the display control device comprising:

a controller including a voice recognition unit configured to detect a voice input of the occupant and recognize the voice by voice processing of a detected voice; and

a display control unit configured to control the display to display a main screen including an operating state image indicating an operating state of a dialogue system, to display a category screen including at least one of a name of the category and an icon of the category in which commands are grouped, and to display a list screen including a list grouping a plurality of syntaxes having at least commands,

the controller being configured to

control the display to display the main screen before the voice recognition unit detects the voice,

control the display to display the category screen when the voice processing is not completed normally in the main screen, and

change the display screen from the category screen to the list screen when a category is selected in a state in which the category screen is displayed.

17 . The display control device according to claim 16 , wherein

the controller being further configured to determine that the voice processing is not completed normally upon determining that the voice input of the occupant is not received within a predetermined amount of time after displaying the main screen or upon determining occurrence of an abnormality during the voice processing.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 11, 2024
From: TASAKI, MASARU; SONEZAKI, EIJI
To: NISSAN MOTOR CO., LTD.
Reel/Frame 067683/0430 →
Continuity (1)
Related Publication 20240265918A1 · Aug 8, 2024
References Cited (12)
US 20140350925A1 · Park · 2014 [cited by examiner]
US 20180025728A1 · Bae · 2018 [cited by examiner]
US 20190012137A1 · Lim · 2019 [cited by examiner]
US 20190035399A1 · Chakladar · 2019 [cited by examiner]
US 20220057989A1 · Watanabe · 2022 [cited by examiner]
US 20230350707A1 · Kim · 2023 [cited by examiner]
JP 2002168643A · 2002 [cited by applicant]
JP 201325605A · 2013 [cited by applicant]
JP 2020144274A · 2020 [cited by applicant]
WO 2016002406A1 · 2016 [cited by applicant]
WO 2020137607A1 · 2020 [cited by applicant]
Extended European Search Report of Mar. 22, 2024 of corresponding European Patent Application No. 21943348.9. [cited by applicant]