IP Library Granted Patent US 11,270,703
Granted Patent B2
US 11,270,703 · App. 16/795,849 · Granted Mar 8, 2022

Audio firewall

Inventors: Philip Alan Bunker (Vista, CA); Mayank Saxena (Pleasanton, CA)
Assignee: Nortek Security & Control LLC
G10L15/26G10L13/00G10L15/22G10L15/30G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,270,703
App. No.
16/795,849
Granted
Mar 8, 2022
Kind
B2
Abstract

An audio firewall system has a microphone that generates audio data. A speech-to-text engine converts the audio data to text data. The text data is parsed for a service wake word and corresponding content data. The service wake word identifies one of a local security system and a remote assistant server. A text-to-speech engine converts the service wake word and the corresponding content data to converted audio data. The converted audio data is provided to the remote assistant server. The content data is provided to the local security system. The audio firewall system receives a response from the remote assistant server or the local security system and outputs an audio signal corresponding to the response.

Claims (59)

1. An audio firewall system comprising:

a microphone configured to generate first audio data;

a speech-to-text engine configured to convert the first audio data to text data;

a service engine configured to parse the text data for a service wake word and a request, to determine whether the first audio data is directed to a local security system based on the request, and to determine whether the first audio data is directed to a remote assistant server based on the wake word;

an audio anonymizer coupled to the service engine, the audio anonymizer configured to convert the service wake word and the request in the text data to second audio data in response to determining that the first audio data is directed to the remote assistant server based on the wake word;

a remote service interface configured to provide the second audio data to the remote assistant server in response to determining that the first audio data is directed to the remote assistant server based on the wake word; and

a local security system interface configured to provide the request in the text data to the local security system in response to determining that the first audio data is directed to the local security system based on determining that the request identifies a local device that is locally connected to the local security system, and the request includes a command to control the local device.

2. The audio firewall system of claim 1 , wherein the remote service interface is configured to receive a response from the remote assistant server in response to providing the second audio data to the remote assistant server, and further comprising:

a speaker configured to output an audio signal corresponding to the response.

3. The audio firewall system of claim 1 , wherein the local security system interface is configured to receive a response from the local security system in response to providing the request to the local security system, and further comprising:

a speaker configured to output an audio signal corresponding to the response.

4. The audio firewall system of claim 1 , wherein the audio anonymizer is further configured to adjust at least one of a pitch or a speed of the first audio data.

5. The audio firewall system of claim 1 , wherein the service engine is configured to identify the service wake word from a plurality of service wake words, and to identify the remote assistant server from a plurality of remote assistant servers, the remote assistant server corresponding to the service wake word, each remote assistant server identified with a corresponding service wake word.

6. The audio firewall system of claim 5 , wherein the service engine is configured to receive a custom service wake word, to determine that the custom service wake word is different from the plurality of service wake words from the plurality of remote assistant servers, and to associate the custom service wake word with the local security system in response to determining that the custom service wake word is different from the plurality of service wake words.

7. The audio firewall system of claim 1 , wherein the service engine is configured to identify the service wake word, and to identify the local security system corresponding to the service wake word.

8. The audio firewall system of claim 1 , wherein the remote service interface is configured to receive a response from the remote assistant server in response to the request.

9. The audio firewall system of claim 1 , wherein the remote service interface is configured to communicate with a plurality of remote assistant servers, each remote assistant server having a corresponding service wake word.

10. The audio firewall system of claim 1 , wherein the local security system is configured to identify the local device connected to the local security system based on the request, to generate a local command to the local device based on the request and to receive a response from the local device, and

wherein the audio firewall system further comprises a speaker configured to generate an audio signal corresponding to the response from the local device.

11. A method comprising:

generating first audio data with a microphone of an audio firewall system;

converting the first audio data to text data;

parsing the text data for a service wake word and a request;

determining whether the first audio data is directed to a local security system based on the request by determining that the request identifies a local device that is locally connected to a local security system, and that the request includes a command to control the local device;

determining whether the first audio data is directed to a remote assistant server based on the wake word;

in response to determining that the first audio data is directed to the remote assistant server based on the wake word, converting the service wake word and the request in the text data to second audio data, and providing the second audio data to the remote assistant server; and

in response to determining that the first audio data is directed to the local security system, providing the request in the text data to the local security system.

12. The method of claim 11 , further comprising:

receiving a response from the remote assistant server in response to providing the second audio data to the remote assistant server; and

outputting an audio signal corresponding to the response with a speaker at the audio firewall system.

13. The method of claim 11 , further comprising:

receiving a response from the local security system in response to providing the request to the local security system; and

outputting an audio signal corresponding to the response with a speaker at the audio firewall system.

14. The method of claim 11 , further comprising:

adjusting at least one of a pitch or a speed of the first audio data before converting the first audio data to text data.

15. The method of claim 11 , further comprising:

identifying the service wake word from a plurality of service wake words, in the text data; and

identifying the remote assistant server from a plurality of remote assistant servers, the remote assistant server corresponding to the service wake word, each remote assistant server identified with a corresponding service wake word.

16. The method of claim 15 , further comprising:

receiving a custom service wake word;

determining that the custom service wake word is different from the plurality of service wake words from the plurality of remote assistant servers; and

associating the custom service wake word with the local security system in response to determining that the custom service wake word is different from the plurality of service wake words.

17. The method of claim 11 , further comprising:

identifying the service wake word; and

identifying the local security system corresponding to the service wake word.

18. The method of claim 11 , wherein the audio firewall system is configured to communicate with a plurality of remote assistant servers, each remote assistant server having a corresponding service wake word.

19. The method of claim 11 , further comprising:

identifying Rail the local device connected to the local security system based on the request;

generating a command to the local device based on the request;

receiving a response from the local device; and

generating an audio signal corresponding to the response from the local device.

20. A non-transitory machine-storage medium storing instructions that, when executed by one or more processors of a machine, cause the one or more processors to perform operations comprising:

generating first audio data with a microphone of an audio firewall system;

converting the first audio data to text data;

parsing the text data for a service wake word and a request;

determining whether the first audio data is directed to a local security system based on the request by determining that the request identifies a local device that is locally connected to a local security system, and that the request includes a command to control the local device;

determining whether the first audio data is directed to a remote assistant server based on the wake word;

in response to determining that the first audio data is directed to the remote assistant server based on the wake word, converting the service wake word and the request in the text data to second audio data, and providing the second audio data to the remote assistant server; and

in response to determining that the first audio data is directed to the local security system, providing the request in the text data to the local security system.

Assignments (2)
CHANGE OF NAME Recorded Jan 9, 2024
From: NORTEK SECURITY & CONTROL LLC
To: NICE NORTH AMERICA LLC
Reel/Frame 066242/0513 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2020
From: BUNKER, PHILIP ALAN; SAXENA, MAYANK
To: NORTEK SECURITY & CONTROL LLC
Reel/Frame 052589/0463 →