IP Library › Granted Patent US 10,614,807
Granted Patent B2
US 10,614,807 · App. 16/178,122 · Granted Apr 7, 2020

Arbitration-based voice recognition

Inventors: Steven Beckhardt (Boston, MA); Ted Lin (Ayer, MA)
Assignee: Sonos, Inc.
G10L15/22G06F3/165G06F3/167G10L15/32G10L15/30G10L2015/088G10L2015/223H04R27/00H04R2227/003H04R2227/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,614,807
App. No.
16/178,122
Granted
Apr 7, 2020
Kind
B2
Abstract

A system comprises a first network microphone device (NMD) communicatively coupled to a second NMD via a network interface of the first NMD, wherein the first NMD is configured to: receive, from the second NMD via the network interface of the first NMD, an arbitration message comprising (i) a first measure of confidence associated with a voice input detected by the second NMD and (ii) an identifier associated with at least a portion of the voice input detected by the second NMD, determine that the first measure of confidence is greater than a second measure of confidence, wherein the second measure of confidence is associated with a voice input detected by the first NMD via at least one microphone of the second NMD, and based on the determination, perform voice recognition based on the voice input detected by the second NMD, wherein the voice input detected by the second NMD comprises a command to control the playback of the audio content by at least one of the first or the second NMDs.

Claims (54)

1. A first network microphone device (NMD) comprising:

one or more audio processing components configured to cause the first NMD to play back audio content;

a network interface configured to communicatively couple the first NMD to a second NMD that is configured to play back audio content;

one or more processors;

at least one microphone; and

a tangible, non-transitory, computer-readable medium having instructions stored thereon that are executable by the one or more processors to cause the first NMD device to:

while operating in a mode in which the first NMD is configured to play back audio content in synchrony with the second NMD:

detect a voice input via the at least one microphone;

determine a first measure of confidence associated with the voice input detected by the first NMD;

receive, from the second NMD via the network interface, an arbitration message comprising (i) a second measure of confidence associated with a voice input detected by the second NMD and (ii) the voice input detected by the second NMD;

determine that the second measure of confidence is greater than the first measure of confidence;

based on the determination, perform voice recognition based on the voice input detected by the second NMD, wherein the voice input detected by the second NMD comprises a command to control the synchronous playback of the audio content by the first NMD and the second NMD; and

after performing voice recognition based on the voice input detected by the second NMD, executing the command to control the synchronous playback of the audio content by the first NMD and the second NMD.

2. The first NMD of claim 1 , wherein the arbitration message further comprises voice data that is based on the voice input detected by the second NMD, and wherein the instructions that are executable to cause the first NMD to perform voice recognition further comprise instructions that are executable to cause the first NMD to:

transmit a voice message that comprises the voice data to a server via a network for voice processing.

3. The first NMD of claim 2 , wherein the voice data is indicative of a wakeword.

4. The first NMD of claim 2 , further comprising instructions that are executable to cause the first NMD to:

receive, from the server, a voice response to the voice message and play back the voice response.

5. The first NMD of claim 2 , wherein the voice message is a first voice message, and wherein the arbitration message further comprises a value indicating an interval of time that the second NMD will wait before transmitting, to the server, a second voice message that is based on the voice input detected by the second NMD.

6. The first NMD of claim 1 , wherein the arbitration message comprises a header that further comprises (i) voice data that is based on the voice input detected by the second NMD, (ii) an identifier associated with a source of the voice input detected by the second NMD, and (iii) a timestamp value indicating a time at which the arbitration message was transmitted by the second NMD.

7. The first NMD of claim 1 , wherein the first NMD is configured to, via the one or more audio processing components, play back a first audio channel of the audio content in synchrony with playback of a second audio channel of the audio content by the second NMD.

8. The first NMD of claim 1 , wherein the voice input detected by the second NMD comprises one or more of (i) a wakeword or (ii) at least one command.

9. A non-transitory computer-readable storage medium storing instructions thereon that, when executed, cause at least one processor of a first network microphone device (NMD) to:

while operating in a mode in which the first NMD is configured to play back audio content in synchrony with the second NMD:

detect a voice input via the at least one microphone;

determine a first measure of confidence associated with the voice input detected by the first NMD;

receive, from the second NMD via a network interface of the first NMD, an arbitration message comprising (i) a second measure of confidence associated with a voice input detected by the second NMD and (ii) the voice input detected by the second NMD;

determine that the second measure of confidence is greater than the first measure of confidence;

based on the determination, perform voice recognition based on the voice input detected by the second NMD, wherein the voice input detected by the second NMD comprises a command to control the synchronous playback of the audio content by the first NMD and the second NMD; and

after performing voice recognition based on the voice input detected by the second NMD, executing the command to control the synchronous playback of the audio content by the first NMD and the second NMD.

10. The non-transitory computer-readable storage medium of claim 9 , wherein the arbitration message further comprises voice data that is based on the voice input detected by the second NMD, and wherein the instructions that are executable to cause the first NMD to perform voice recognition further comprise instructions that are executable to cause the first NMD to:

transmit a voice message that comprises the voice data to a server via a network for voice processing.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the voice data is indicative of a wakeword.

12. The non-transitory computer-readable storage medium of claim 10 , further comprising instructions that are executable to cause the first NMD to:

receive, from the server, a voice response to the voice message and play back the voice response.

13. The non-transitory computer-readable medium of claim 10 , wherein the voice message is a first voice message, and wherein the arbitration message further comprises a value indicating an interval of time that the second NMD will wait before transmitting, to the server, a second voice message that is based on the voice input detected by the second NMD.

14. The non-transitory computer-readable storage medium of claim 9 , wherein the arbitration message comprises a header that further comprises (i) voice data that is based on the voice input detected by the second NMD, (ii) an identifier associated with a source of the voice input detected by the second NMD, and (iii) a timestamp value indicating a time at which the arbitration message was transmitted by the second NMD.

15. The non-transitory computer-readable storage medium of claim 9 , further comprising instructions that, when executed, cause the at least one processor to:

via one or more audio processing components of the first NMD, play back a first audio channel of the audio content in synchrony with playback of a second audio channel of the audio content by the second NMD.

16. The non-transitory computer-readable medium of claim 9 , wherein the voice input detected by the second NMD comprises one or more of (i) a wakeword or (ii) at least one command.

17. A system comprising:

a first network microphone device (NMD) communicatively coupled to a second NMD via a network interface of the first NMD, wherein the first NMD is configured to:

while operating in a mode in which the first NMD is configured to play back audio content in synchrony with the second NMD:

detect a voice input via the at least one microphone;

determine a first measure of confidence associated with the voice input detected by the first NMD;

receive, from the second NMD via the network interface of the first NMD, an arbitration message comprising (i) a second measure of confidence associated with a voice input detected by the second NMD and (ii) the voice input detected by the second NMD;

determine that the second measure of confidence is greater than the first measure of confidence;

based on the determination, perform voice recognition based on the voice input detected by the second NMD, wherein the voice input detected by the second NMD comprises a command to control the synchronous playback of the audio content by the first NMD and the second NMD; and

after performing voice recognition based on the voice input detected by the second NMD, executing the command to control the synchronous playback of the audio content by the first NMD and the second NMD.

18. The system of claim 17 , wherein the arbitration message further comprises voice data that is based on the voice input detected by the second NMD, and wherein to perform voice recognition, the first NMD is configured to:

transmit a voice message that comprises the voice data to a server via a network for voice processing.

19. The system of claim 18 , wherein the first NMD is further configured to:

receive, from the server, a voice response to the voice message and play back the voice response.

20. The system of claim 17 , wherein the voice input detected by the second NMD comprises one or more of (i) a wakeword or (ii) at least one command.

Assignments (2)
SECURITY AGREEMENT Recorded Oct 15, 2021
From: SONOS, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 058123/0206 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 10, 2018
From: BECKHARDT, STEVEN; LIN, TED
To: SONOS, INC.
Reel/Frame 047724/0666 →
Continuity (2)
Continuation 15297627 · Oct 19, 2016
Related Publication 20190074008A1 · Mar 7, 2019
Cited By (19)
US 12,211,490 US 12,217,748 US 12,230,291 US 12,236,932 US 12,283,269 US 12,284,417 US 12,327,549 US 12,327,556 US 12,360,734 US 12,387,716 US 12,424,220 US 12,505,832 US 12,513,479 US 12,518,756 US 12,579,978 US 12,699,543 US 12,711,962 US 12,732,547 US 12,748,566