IP Library Granted Patent US 10,477,294
Granted Patent B1
US 10,477,294 · App. 15/883,663 · Granted Nov 12, 2019

Multi-device audio capture

Inventors: Milos Jorgovanovic (Mountain View, CA); Hans Edward Birch-Jensen (San Jose, CA)
Assignee: AMAZON TECHNOLOGIES, INC.
H04R1/1016G06F3/167G10L15/08G10L15/20H04R1/083G10L2015/088H04R2420/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,477,294
App. No.
15/883,663
Granted
Nov 12, 2019
Kind
B1
Abstract

A system and method includes a first device (e.g., a primary or primary earbud) that detects first audio corresponding to an utterance and a second device (e.g., a secondary or secondary earbud) that detects second audio corresponding to the utterance. The first device determines first audio data corresponding to the first audio and the second device determines second audio data corresponding to the second audio. The first device determines a first quality metric corresponding to the first audio data, and the second device determines a second quality metric corresponding to the second audio data. The second device transmits the second quality metric to first device, which compares it to the first quality metric. If the first device determines the second quality metric is better, it transmits the second audio data to a third device (e.g., a smartphone) and enters a power-saving state.

Claims (87)

1. A computer-implemented method comprising:

detecting first audio using a primary wireless earbud, wherein the primary wireless earbud:

comprises a first set of microphones,

is wirelessly connected to a secondary wireless earbud over a first wireless communication connection, and

is wirelessly connected to a mobile device over a second wireless communication connection;

determining, by the primary wireless earbud, first audio data corresponding to the first audio;

determining, by the primary wireless earbud, that the first audio data corresponds to speech;

determining, by the primary wireless earbud, a first signal-to-noise ratio (SNR) corresponding to the first audio data;

receiving, by the primary wireless earbud from the secondary wireless earbud:

first data corresponding to the secondary wireless earbud detecting speech in second audio data corresponding to second audio detected by the secondary wireless earbud, and

a second SNR corresponding to the second audio data;

determining that the second SNR is higher than the first SNR;

receiving, by the primary wireless earbud from the secondary wireless earbud, at least a portion of the second audio data; and

sending, by the primary wireless earbud, at least the portion of the second audio data to the mobile device.

2. The method of claim 1 , further comprising, prior to sending the second audio data, at least one of:

determining, by the primary wireless earbud, that the first audio data includes a wakeword; or

determining, by the secondary wireless earbud, that the second audio data includes the wakeword.

3. A computer-implemented method comprising:

detecting, by a first earbud of a pair of earbuds, first audio corresponding to a first utterance;

determining, by the first earbud, first audio data corresponding to the first audio;

determining, by the first earbud, a first quality metric corresponding to the first audio data;

receiving, by the first earbud and from a second earbud of the pair of earbuds, a second quality metric corresponding to second audio data corresponding to the first utterance;

determining that the second quality metric corresponds to higher audio quality than the first quality metric;

receiving, by the first earbud and from the second earbud, at least a portion of the second audio data;

sending, by the first earbud and to a user device, output audio data comprising at least the portion of the second audio data; and

causing at least a portion of the first earbud to enter a power-saving state.

4. The method of claim 3 , further comprising:

receiving, by the first earbud from the second earbud, first data corresponding to the second earbud detecting second audio corresponding to the second audio data; and

sending, from the first earbud to the second earbud, a command to send the second audio data.

5. The method of claim 3 , further comprising, prior to sending the output audio data:

processing, by the first earbud, the first audio data to detect a representation of a wakeword; and

receiving, from the second earbud, an indication of detection of the wakeword by the second earbud.

6. The method of claim 3 , further comprising:

determining that the first quality metric is greater than a threshold; and

configuring the output audio data to further comprise at least a portion of the first audio data.

7. The method of claim 3 , further comprising:

determining, using the first earbud, noise data common to the first audio data and the second audio data; and

removing the noise data from the first audio data and the second audio data to determine filtered audio data,

wherein the output audio data comprises the filtered audio data.

8. The method of claim 3 , further comprising:

determining that the first quality metric is lower than a threshold; and

causing at least the portion of the first earbud to enter the power-saving state based at least in part on the first quality metric being lower than the threshold.

9. The method of claim 3 , wherein causing at least the portion of the first earbud to enter the power-saving state further comprises at least one of:

causing a voice-activity detection component to cease analyzing the first audio data; or

causing a wakeword detection component to cease analyzing the first audio data.

10. The method of claim 3 , wherein determining the first audio data further comprises at least one of:

determining, by the first earbud, the first audio data comprises a voice; or

receiving, by the first earbud from the second earbud, first data corresponding to the second earbud determining the second audio data comprises the voice.

11. The method of claim 10 , wherein determining the second quality metric corresponds to higher audio quality than the first quality metric further comprises at least one of:

determining, by the first earbud, that the first audio data comprises a wakeword; or

receiving, by the first earbud from the second earbud, second data corresponding to the second earbud determining the second audio data comprises the wakeword.

12. A system comprising:

at least one processor; and

at least one memory including instructions that, when executed by the at least one processor, cause the system to:

detect, by a first earbud, first audio corresponding to an utterance,

determine, by the first earbud, first audio data corresponding to the first audio;

determine, by the first earbud, a first quality metric corresponding to the first audio data,

receive, by the first earbud and from a second earbud, a second quality metric corresponding to second audio data corresponding to the first utterance,

determine that the second quality metric corresponds to higher audio quality than the first quality metric,

receive, by the first earbud from the second earbud, at least a portion of the second audio data,

send transmit, by the first earbud to a user device, output audio data comprising at least the portion of the second audio data, and

cause at least a portion of the first earbud to enter a power-saving state.

13. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:

receive, by the first earbud from the second earbud, first data corresponding to the second earbud detecting second audio corresponding to the second audio data; and

send, from the first earbud to the second earbud, a command to send the second audio data.

14. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:

process, by the first earbud, the first audio data to detect a representation of a wakeword; and

receive, from the second earbud, an indication of detection of the wakeword by the second earbud.

15. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:

determine that the first quality metric is greater than a threshold; and

configure the output audio data to further comprise at least a portion of the first audio data.

16. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:

determine, using the first earbud, noise data common to the first audio data and the second audio data; and

remove the noise data from the first audio data and the second audio data to determine filtered audio data,

wherein the output audio data comprises the filtered audio data.

17. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to:

determine that the first quality metric is lower than a threshold; and

cause at least the portion of the first earbud to enter the power-saving state based at least in part on the first quality metric being lower than the threshold.

18. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to do at least one of:

cause a voice-activity detection component to cease analyzing the first audio data; or

cause a wakeword detection component to cease analyzing the first audio data.

19. The system of claim 12 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to do at least one of:

determine, by the first earbud, the first audio data comprises a voice; or

receive, by the first earbud from the second earbud, first data corresponding to the second earbud determining the second audio data comprises the voice.

20. The system of claim 19 , further comprising additional instructions that, when executed by the at least one processor, further cause the system to do at least one of:

determine, by the first earbud, that the first audio data comprises a wakeword; or

receive, by the first earbud from the second earbud, second data corresponding to the second earbud determining the second audio data comprises the wakeword.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 30, 2018
From: JORGOVANOVIC, MILOS; BIRCH-JENSEN, HANS EDWARD
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 044772/0857 →
Cited By (3)
US 12,190,865 US 12,348,931 US 12,413,921