IP Library › Granted Patent US 11,354,089
Granted Patent B2
US 11,354,089 · App. 15/374,595 · Granted Jun 7, 2022

System and method for dialog interaction in distributed automation systems

Inventors: Leah Nicolich-Henkin (Pittsburgh, PA); Cory Henson (Pittsburgh, PA); Joao P. Sousa (Pittsburgh, PA)
Assignee: Robert Bosch GmbH
G06F3/167H04L12/282H04L12/2827G10L13/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,354,089
App. No.
15/374,595
Filed
Dec 9, 2016
Granted
Jun 7, 2022
Kind
B2
Art Unit
2173
USPC
715/728
Abstract

A method for generating a user interface with a user interface device in a distributed automation system includes receiving a service message from a home automation device in the distributed automation system, identifying a state of a dialog manager of the user interface device in response to receiving the service message, and generating a natural language output message based at least in part on a device identifier parameter in the service message and a plurality of natural language templates stored in the memory in response to the dialog manager being in an idle state. The method further includes storing the service message in a priority queue in the memory based on a priority level parameter corresponding to the service message in response to the dialog manager being in an active state.

Claims (60)

1. A method for dialog interaction with a user interface in a distributed automation system comprising:

receiving with a network interface device in a user interface device, a service message from a home automation device in the distributed automation system;

identifying, with the processor, a state of a dialog manager that the processor executes with stored program instructions stored in a memory of the user interface device in response to receiving the service message;

generating, with the processor and an output device, a natural language output message based at least in part on a device identifier parameter in the service message and a plurality of natural language templates stored in the memory in response to the dialog manager being in an idle state;

storing, with the processor, the service message in a priority queue in the memory based on a priority level parameter corresponding to the service message in response to the dialog manager being in an active state

identifying, with the processor, a first room location of the user interface device;

identifying, with the processor, a second room location of the home automation device based on the service message; and

increasing, with the processor, the priority level of the priority level parameter in the service message in response to the first room location and the second room location corresponding to a single room.

2. The method of claim 1 further comprising:

generating, with the processor and a display device associated with the user interface device, a text output message based on the natural language output message.

3. The method of claim 1 further comprising:

generating, with the processor and an audio output device associated with the user interface device, a speech synthesized output message based on the natural language output message.

4. The method of claim 1 further comprising:

identifying, with the processor, a state transition in the dialog manager from the active state to the idle state; and

generating, with the processor and the output device, another natural language output message based at least in part on a device identifier parameter in a service message stored in the priority queue having a highest priority level and the plurality of natural language templates stored in the memory in response to the state transition of the dialog manager.

5. The method of claim 1 further comprising:

identifying, with the processor, that the priority level parameter in the service message corresponds to a predetermined priority level for interruption of the dialog manager in the active state; and

generating, with the processor and the output device, the natural language output message while the dialog manager is in the active state in response to the priority level parameter in the service message corresponding to the predetermined priority level.

6. The method of claim 1 further comprising:

receiving, with an audio input device associated with the user interface device, speech input from the user in response to the natural language output message;

identifying, with the processor and the dialog manager, at least one parameter in the speech input that corresponds to a predetermined command template stored in the memory in association with the device identifier parameter of the home automation device that produced the service message;

generating, with the processor and the dialog manager, a command message based on the at least one parameter and the predetermined command template; and

transmitting, with the network interface device, the command message to the home automation device to control operation of the home automation device based on the speech input from the user.

7. The method of claim 6 further comprising:

generating, with the processor, the dialog manager, and the output device, an output request for an additional parameter in the predetermined command template for the command message in response to the speech input from the user not including the additional parameter.

8. The method of claim 1 further comprising:

decreasing, with the processor, the priority level of the of the priority level parameter in the service message in response to the first room location and the second room location corresponding to different rooms.

9. A user interface device in a distributed automation system comprising:

a network interface device configured to receive service messages from a plurality of home automation devices;

an output device;

a memory; and

a processor operatively connected to the network interface device, the output device, and the memory, the processor being configured to:

receive a service message from a home automation device in the distributed automation system with the network interface device;

identify a state of a dialog manager that the processor executes with stored program instructions stored in the memory of the user interface device in response to receiving the service message;

generate a natural language output message based at least in part on a device identifier parameter in the service message and a plurality of natural language templates stored in the memory in response to the dialog manager being in an idle state with the output device;

store the service message in a priority queue in the memory based on a priority level parameter corresponding to the service message in response to the dialog manager being in an active state;

identify a first room location of the user interface device;

identify a second room location of the home automation device based on the service message; and

increase the priority level of the priority level parameter in the service message in response to the first room location and the second room location corresponding to a single room.

10. The user interface device of claim 9 , wherein the output device is a display device, the processor being further configured to:

generate a text output message based on the natural language output message with the display device.

11. The user interface device of claim 9 , wherein the output device is an audio output device, the processor being further configured to:

generate a speech synthesized output message based on the natural language output message with the audio output device.

12. The user interface device of claim 9 , the processor being further configured to:

identify a state transition in the dialog manager from the active state to the idle state; and

generate another natural language output message with the output device based at least in part on a device identifier parameter in a service message stored in the priority queue having a highest priority level and the plurality of natural language templates stored in the memory in response to the state transition of the dialog manager.

13. The user interface device of claim 9 , the processor being further configured to:

identify that the priority level parameter in the service message corresponds to a predetermined priority level for interruption of the dialog manager in the active state; and

generate the natural language output message with the output device while the dialog manager is in the active state in response to the priority level parameter in the service message corresponding to the predetermined priority level.

14. The user interface device of claim 9 further comprising:

an audio input device; and

the processor being operatively connected to the audio input device and further configured to:

receive speech input from the user with the audio input device in response to the natural language output message;

identify at least one parameter in the speech input that corresponds to a predetermined command template stored in the memory in association with the device identifier parameter of the home automation device that produced the service message;

generate a command message with the dialog manager based on the at least one parameter and the predetermined command template; and

transmit the command message to the home automation device with the network interface device to control operation of the home automation device based on the speech input from the user.

15. The user interface device of claim 14 , the processor being further configured to:

generate an output request for an additional parameter in the predetermined command template for the command message with the dialog manager and the output device in response to the speech input from the user not including the additional parameter.

16. The user interface device of claim 9 , the processor being further configured to:

decrease the priority level of the of the priority level parameter in the service message in response to the first room location and the second room location corresponding to different rooms.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 11, 2018
From: NICOLICH-HENKIN, LEAH; HENSON, CORY; SOUSA, JOAO P.
To: ROBERT BOSCH GMBH
Reel/Frame 044599/0195 →
Continuity (1)
Related Publication 20180165061A1 · Jun 14, 2018
Cited By (1)
US 12,580,788