IP Library Granted Patent US 10,959,029
Granted Patent B2
US 10,959,029 · App. 15/989,715 · Granted Mar 23, 2021

Determining and adapting to changes in microphone performance of playback devices

Inventors: Kurt Thomas Soto (Ventura, CA); Connor Kristopher Smith (New Hudson, MI); Roberto Maria Dizon (Somerville, MA); Patrick Kevin McPherson (Somerville, MA); Charles Conor Sleith (Waltham, MA)
Assignee: Sonos, Inc.
H04R29/005G10L15/08G10L15/22H04R1/406H04R3/005H04R3/04G10L2015/088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,959,029
App. No.
15/989,715
Granted
Mar 23, 2021
Kind
B2
Abstract

Systems and methods for determining and adapting to changes in microphone performance of playback devices are disclosed herein. In one example, an audio input is received at an array of individual microphones of a network microphone device. Output microphone signals are generated from each of the individual microphones based on the audio input. The output microphone signals are analyzed to detect a trigger event. After detecting the trigger event, the output microphone signals are compared to detect aberrant behavior of one or more of the microphones. Optionally, corrective actions can be taken or suggested based on the detection of aberrant behavior of one or more microphones.

Claims (47)

1. A method, comprising:

receiving an audio input via a microphone array of a network microphone device, the microphone array comprising a plurality of individual microphones;

producing output microphone signals from each of the individual microphones based on the audio input;

analyzing the output microphone signals to detect a wake word associated with a voice assistant service (VAS);

deriving frequency-domain microphone performance data for each of the individual microphones based on the output microphone signals;

capturing a voice input based on at least one of the output microphone signals; and

after detecting the wake word, (i) sending the voice input to one or more computing devices associated with the VAS, and (ii) sending the frequency-domain microphone performance data, without exposing the audio input from which the microphone performance data is derived, to a remote evaluator to detect aberrant behavior of one or more of the individual microphones, the remote evaluator being distinct from the VAS;

receiving an indication of the aberrant behavior of one or more of the individual microphones; and

modifying a beamforming algorithm of the network microphone device to at least partially compensate for the aberrant behavior.

2. The method of claim 1 , further comprising, at the remote evaluator, comparing the frequency-domain microphone performance data to detect the aberrant behavior.

3. The method of claim 2 , wherein comparing the frequency-domain microphone performance data comprises:

analyzing a frequency response for each of the individual microphones; and

comparing the frequency responses for each of the individual microphones.

4. The method of claim 3 , wherein comparing the frequency responses comprises:

determining a mean or median frequency response for each of the individual microphones; and

identifying any microphone for which the frequency response deviates from the mean or median frequency response by more than a threshold amount.

5. The method of claim 1 , wherein the microphone performance data comprises a windowed average of frequency response data in predefined bands within a frequency range.

6. The method of claim 5 , wherein the frequency range is between about 100 Hz and about 8 kHz.

7. A non-transitory computer-readable medium comprising instructions for identifying aberrant microphone behavior, the instructions, when executed by a processor, causing the processor to perform the following operations:

receiving output microphone signals from individual microphones of a network microphone device;

analyzing the output microphone signals to detect a wake word associated with a voice assistant service (VAS);

deriving frequency-domain microphone performance data for each of the individual microphones based on the output microphone signals;

capturing a voice input based on at least one of the output microphone signals;

after detecting the wake word, (i) sending the voice input to one or more computing devices associated with the VAS, and (ii) sending the frequency-domain microphone performance data, without exposing the output microphone signals from which the microphone performance data is derived, to a remote evaluator to detect aberrant behavior of one or more of the individual microphones, the remote evaluator being distinct from the VAS;

receiving, from the remote evaluator, an indication of the aberrant behavior of one or more of the individual microphones; and

performing corrective action to at least partially compensate for the aberrant behavior.

8. The computer-readable medium of claim 7 , wherein the operations further comprise, after detecting the wake word:

receiving additional audio input via the plurality of individual microphones; and

passing the additional audio input to the one or more computing devices associated with the VAS.

9. The computer-readable medium of claim 7 , wherein the microphone performance data comprises a windowed average of frequency response data in predefined bands within a frequency range.

10. The computer-readable medium of claim 9 , wherein the frequency range is between about 100 Hz and about 8 kHz.

11. A network microphone device comprising:

one or more processors;

a microphone array comprising a plurality of individual microphones; and

a computer-readable medium storing instructions that, when executed by the one or more processors, cause the network microphone device to perform operations, the operations comprising:

receiving output microphone signals from each of the microphones;

analyzing the output microphone signals to detect a wake word associated with a voice assistant service (VAS);

deriving frequency-domain microphone performance data for each of the individual microphones based on the output microphone signals;

capturing a voice input based on at least one of the output microphone signals;

after detecting the wake word, (i) sending the voice input to one or more computing devices associated with the VAS, and (ii) sending the frequency-domain microphone performance data, without exposing the output microphone signals from which the microphone performance data is derived, to a remote evaluator to detect aberrant behavior of one or more of the individual microphones;

receiving an indication of the aberrant behavior of one or more of the individual microphones; and

modifying a beamforming algorithm of the network microphone device to at least partially compensate for the aberrant behavior.

12. The device of claim 11 , wherein the operations further comprise, after detecting the wake word:

receiving additional audio input via the plurality of individual microphones; and

passing the additional audio input to the one or more computing devices associated with the VAS.

13. The device of claim 11 , wherein the microphone performance data comprises a windowed average of frequency response data in predefined bands within a frequency range.

14. The device of claim 13 , wherein the frequency range is between about 100 Hz and about 8 kHz.

Assignments (2)
SECURITY AGREEMENT Recorded Oct 15, 2021
From: SONOS, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 058123/0206 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 13, 2018
From: SOTO, KURT THOMAS; DIZON, ROBERTO MARIA; MCPHERSON, PATRICK KEVIN; SLEITH, CHARLES CONOR; SMITH, CONNOR KRISTOPHER
To: SONOS, INC.
Reel/Frame 046075/0360 →
Continuity (1)
Related Publication 20190364375A1 · Nov 28, 2019
Cited By (2)
US 12,271,656 US 12,342,149