IP Library Granted Patent US 11,264,020
Granted Patent B2
US 11,264,020 · App. 16/028,126 · Granted Mar 1, 2022

Systems and methods for conversational interaction with a building automation system

Inventors: Sajjad Pourmohammad (Milwaukee, WI); Joseph M. Mueller (Hales Corners, WI); Youngchoon Park (Brookfield, WI); Sudhi R. Sinha (Milwaukee, WI); Daniel Mellenthin (Milwaukee, WI); Gerald A. Asp (Milwaukee, WI)
Assignee: Johnson Controls Tyco IP Holdings LLP
G10L15/22G06F3/167G10L15/1815G10L15/30H04L12/2803G06F3/0482G06F3/0484G06F21/31G06F2221/2141G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,264,020
App. No.
16/028,126
Granted
Mar 1, 2022
Kind
B2
Abstract

A system for user interaction with a building automation system (BAS). The system includes a user device configured to accept vocal inputs and produce audio outputs, and a server in communication with the user device and a BAS controller. The server is configured to translate a vocal input to input text, and determine and store a conversation context based on the input text. The server is further configured to determine a desired output corresponding to the input text. Additionally, the server is configured to receive, from the BAS controller, BAS data corresponding to the desired output, and generate at least one audio output from the BAS data. The user device outputs the at least one audio output, and the server is configured to apply the conversation context to a subsequent vocal input.

Claims (60)

1. A system for user interaction with a building automation system (BAS), the system comprising:

a user device configured to accept vocal inputs and produce audio outputs; and

a server in communication with the user device and a BAS controller, the server configured to:

query the BAS controller using one or more application programming interfaces (API) of the BAS controller with an element associated with input text based on vocal input, the query configured to cause the BAS controller to return an operational parameter stored by the BAS controller;

receive, from the BAS controller, BAS data including the operational parameter associated with a BAS device or a user corresponding to desired output, wherein the BAS data corresponds to a detected equipment fault;

generate at least one audio output related to the operational parameter from the BAS data, wherein the at least one audio output alerts a user of the detected equipment fault; and

submit a maintenance request based on the detected equipment fault.

2. The system of claim 1 , further comprising updating a BAS parameter based on a desired output associated with the input text.

3. The system of claim 2 , wherein the vocal input corresponds to a request to add a new user, and updating the BAS parameter comprises adding the new user to a BAS database and providing corresponding access rights.

4. The system of claim 1 , wherein the maintenance request comprises at least an equipment name, a location, and fault data.

5. The system of claim 1 , wherein the maintenance request comprises an order and payment for a replacement part.

6. The system of claim 1 , wherein the BAS data corresponds to a detected equipment fault, the server further configured to:

determine a plurality of potential fault causes;

calculate a probability for each potential fault cause; and

determine the potential fault with a highest probability;

wherein the at least one audio output corresponds to the potential fault with the highest probability.

7. A method of user interaction with a building automation system (BAS), the method comprising:

query a BAS controller using one or more application programming interfaces (API) of the BAS controller with an element associated with first user input, the query configured to cause the BAS controller to return an operational parameter stored by the BAS controller;

receiving BAS data from the BAS controller, the BAS data including the operational parameter associated with a BAS device or a user corresponding to the first user input;

providing, to a user, a first output corresponding to the operational parameter from the BAS data by displaying the first output via a user interface;

displaying a list of selectable options corresponding to the first output

determining a second user input by detecting a user selection corresponding to the second user input;

reprioritizing the list of selectable options for subsequent inputs based on the user selection;

forming a second output based on the second user input and a conversation context of the first user input; and

providing, to the user, the second output by displaying the second output via the user interface.

8. The method of claim 7 , further comprising:

determining a user ID; and

customizing each of the first output and the second output, based on the user ID.

9. The method of claim 8 , further comprising:

determining access rights corresponding to the user ID; and

updating a BAS parameter based on at least one of the first user input and the second user input,

wherein the access rights comprise updating the BAS parameter.

10. The method of claim 7 , further comprising:

accessing a BAS database to determine the BAS data;

comparing the second output to the first output to determine a preferred output corresponding to the first user input; and

updating the BAS database with the preferred output.

11. The method of claim 7 , wherein the conversation context comprises intent and entity parameters corresponding to the first user input.

12. The method of claim 7 , further comprising resetting the conversation context when a new user provides an input.

13. The method of claim 7 , further comprising resetting the conversation context after a threshold amount of time without subsequent user inputs.

14. A user device for communicating with a building automation system (BAS), the user device comprising:

a proximity sensor configured to determine a user location with respect to the user device;

a microphone configured to accept vocal inputs from a user;

a speaker configured to produce audio outputs to the user;

a processor in communication with the proximity sensor, the microphone, and the speaker, and configured to:

receive user location data from the proximity sensor;

receive a vocal input from the microphone;

store, in a memory, context parameters corresponding to the vocal input and the user location data;

determine a desired output corresponding to the vocal input;

query the BAS using one or more application programming interfaces (API) of the BAS with an element associated with the vocal input, the query configured to cause the BAS to return an operational parameter stored by the BAS;

receive BAS data including the operational parameter associated with a BAS device or a user, wherein the BAS data corresponds to a detected equipment fault;

update an operating parameter within the BAS based on the desired output; and

provide an audio output to the speaker, the audio output related to the operational parameter associated with the detected equipment fault from the BAS data;

submit a maintenance request based on the detected equipment fault;

wherein the processor analyzes a subsequent vocal input using the context parameters, prior to providing a subsequent audio output.

15. The user device of claim 14 , wherein the operating parameter corresponds to BAS equipment associated with the user location.

16. The user device of claim 14 , wherein the processor is further configured to:

determine a user ID from the vocal input;

determine access rights corresponding to the user ID; and

verify that the access rights comprise updating the operating parameter.

17. The user device of claim 14 , wherein the vocal input corresponds to a request to add a new user, and updating the operating parameter comprises adding the new user to a BAS database and providing corresponding access rights.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 9, 2024
From: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
To: TYCO FIRE & SECURITY GMBH
Reel/Frame 067056/0552 →
NUNC PRO TUNC ASSIGNMENT Recorded Feb 4, 2022
From: JOHNSON CONTROLS TECHNOLOGY COMPANY
To: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
Reel/Frame 058959/0764 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 8, 2020
From: POURMOHAMMAD, SAJJAD; MUELLER, JOSEPH M.; PARK, YOUNGCHOON; SINHA, SUDHI R.; MELLENTHIN, DANIEL; ASP, GERALD A.
To: JOHNSON CONTROLS TECHNOLOGY COMPANY
Reel/Frame 052345/0524 →