IP Library Granted Patent US 10,079,012
Granted Patent B2
US 10,079,012 · App. 14/692,551 · Granted Sep 18, 2018

Customizing speech-recognition dictionaries in a smart-home environment

Inventor: Gints Valdis Klimanis (Palo Alto, CA)
Assignee: Google LLC
G10L15/065G10L15/07G10L17/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,079,012
App. No.
14/692,551
Filed
Apr 21, 2015
Granted
Sep 18, 2018
Kind
B2
Art Unit
2677
USPC
704/244
Abstract

A method for customizing speech-recognition dictionaries for different smart-home environments may include generating, at a smart-home device mounted in an enclosure, an acoustic impulse response for the enclosure. The method may also include receiving, by the smart-home device, an audio signal captured in the enclosure. The method may additionally include performing, by the smart-home device, a speech-recognition process on the audio signal using a second speech dictionary generated by convolving the acoustic impulse response with a first speech dictionary.

Claims (55)

1. A method for customizing speech-recognition dictionaries for different smart-home environments, the method comprising:

generating, at a smart-home device mounted in an enclosure, an acoustic impulse response for the enclosure;

receiving, by the smart-home device, an audio signal captured in the enclosure;

performing, by the smart-home device, a speech-recognition process on the audio signal using a second speech dictionary that was generated by convolving the acoustic impulse response with a first speech dictionary prior to receiving the audio signal captured in the enclosure, wherein the second speech dictionary comprises a collection of phonemes that are compared to phonemes in the audio signal by the speech-recognition process; and

causing a smart-home system to be controlled based on results of the speech-recognition process on the audio signal.

2. The method of claim 1 , further comprising:

sending the acoustic impulse response from the smart-home device to a management server; and

receiving, by the smart-home device and from the management server, the second speech dictionary, wherein the management server convolved the acoustic impulse response captured by the smart-home device with the first speech dictionary.

3. The method of claim 1 , further comprising:

generating, by the smart-home device, a sound approximating an ideal acoustic impulse;

recording, by the smart-home device, an acoustic response of the enclosure to the sound approximating an ideal acoustic impulse; and

generating the acoustic impulse response for the enclosure from the acoustic response of the enclosure to the sound approximating an ideal acoustic impulse.

4. The method of claim 3 , wherein the acoustic impulse response for the enclosure is generated from the acoustic response of the enclosure.

5. The method of claim 3 , wherein the sound approximating the ideal acoustic impulse comprises an electronic click or chirp generated by the smart-home device.

6. The method of claim 3 , wherein generating the sound approximating the ideal acoustic impulse and recording the acoustic response of the enclosure occur during a learning interval of the smart-home device.

7. The method of claim 6 , wherein the learning interval comprises a time interval within two weeks of an installation of the smart-home device.

8. A smart home device comprising:

a recording device;

one or more environmental sensors;

one or more processors;

one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

generating an acoustic impulse response for the enclosure in which the smart-home device is installed;

receiving, through the recording device, an audio signal captured in the enclosure;

performing, by the one or more processors, a speech-recognition process on the audio signal using a second speech dictionary that was generated by convolving the acoustic impulse response with a first speech dictionary prior to receiving the audio signal captured in the enclosure, wherein the second speech dictionary comprises a collection of phonemes that are compared to phonemes in the audio signal by the speech-recognition process; and

causing a smart-home system to be controlled based on results of the speech-recognition process on the audio signal.

9. The smart-home device of claim 8 , further comprising a wireless network device, wherein the operations further comprise:

sending the acoustic impulse response from the smart-home device to a management server through the wireless network device; and

receiving, from the management server through the wireless network device, the second speech dictionary, wherein the management server convolves the acoustic impulse response captured by the smart-home device with the first speech dictionary to generate the second speech dictionary.

10. The smart-home device of claim 8 , wherein the smart-home device is a hazard detector and the one or more environmental sensors comprises a smoke sensor.

11. The smart-home device of claim 8 , wherein the smart-home device is a thermostat and the one or more environmental sensors comprises a temperature sensor.

12. The smart-home device of claim 8 , wherein the enclosure comprises a room within a home.

13. The smart-home device of claim 8 , wherein the operations further comprise:

receiving input from the one or more environmental sensors indicating that the enclosure is in a first state, wherein the second speech dictionary is used by the speech-recognition process when the enclosure is in the first state.

14. The smart-home device of claim 13 , wherein the operations further comprise:

receiving input from the one or more environmental sensors indicating that the enclosure is in a second state;

generating a second acoustic impulse response for the enclosure in the second state;

receiving, through the recording device, a second audio signal captured in the enclosure while the enclosure is in the second state; and

performing, by the one or more processors, the speech-recognition process on the audio signal using a third speech dictionary generated by convolving the second acoustic impulse response with the first speech dictionary.

15. The smart-home device of claim 14 , wherein:

when the enclosure is in the first state, a door or window of the enclosure is closed; and

when the enclosure is in the second state, the door or window of the enclosure is open.

16. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

generating an acoustic impulse response for the enclosure in which the smart-home device is installed;

receiving, through a recording device, an audio signal captured in the enclosure;

performing, by the one or more processors, a speech-recognition process on the audio signal using a second speech dictionary that was generated by convolving the acoustic impulse response with a first speech dictionary prior to receiving the audio signal captured in the enclosure, wherein the second speech dictionary comprises a collection of phonemes that are compared to phonemes in the audio signal by the speech-recognition process; and

causing a smart-home system to be controlled based on results of the speech-recognition process on the audio signal.

17. The non-transitory, computer-readable medium of claim 16 , wherein the first speech dictionary is stored at the management server and available to a plurality of smart-home devices in a plurality of enclosures.

18. The non-transitory, computer-readable medium of claim 16 , wherein the operations further comprise:

identifying a first user of the smart-home device, wherein the second speech dictionary is further personalized to the first user;

determining that the audio signal captured in the enclosure comprises human speech by the first user; and

using the second speech dictionary for the speech-recognition process in response to determining that the audio signal captured in the enclosure comprises human speech by the first user.

19. The non-transitory, computer-readable medium of claim 16 , wherein the operations further comprise:

transmitting the acoustic impulse response to a second smart-home device in the enclosure over a local smart-home-device network.

20. The non-transitory, computer-readable medium of claim 16 , wherein the operations further comprise:

transmitting the second speech dictionary to a second smart-home device in the enclosure over a local smart-home-device network.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 27, 2015
From: KLIMANIS, GINTS VALDIS
To: GOOGLE INC.
Reel/Frame 035501/0973 →
Continuity (1)
Related Publication 20160314782A1 · Oct 27, 2016
Cited By (1)
US 12,226,913