IP Library Granted Patent US 10,684,817
Granted Patent B2
US 10,684,817 · App. 15/790,012 · Granted Jun 16, 2020

Aggregation of devices based on acoustic monitoring

Inventors: Gaylord Yu (San Francisco, CA); Steven Stupp (Foster City, CA)
Assignee: EVA Automation, Inc.
G06F3/165G06F1/12G11B27/10H04L7/02H04L65/60H04N21/4305H04N21/43615H04N21/8106H04N21/8547H04R27/00H04S7/301H04S7/307H04W4/023H04W56/005H04L29/06027H04R3/12H04R2227/005H04R2420/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,684,817
App. No.
15/790,012
Granted
Jun 16, 2020
Kind
B2
Abstract

An audio/video (A/V) hub that aggregates electronic devices is described. In particular, the A/V hub may measure sound, corresponding to audio content, output by electronic devices (such as electronic devices that include speakers). Then, the A/V hub may aggregate the electronic devices into two or more subsets based on the measured sound. Moreover, the A/V hub may determine, for the subsets, playback timing information, which may specify playback times when the electronic devices in a given subset are to playback the audio content. Next, the A/V hub may transmit, to the electronic devices, one or more frames that include the audio content and playback timing information, where the playback times of the electronic devices in at least the given subset have a temporal relationship so that the playback of the audio content by the electronic devices in the given subset is coordinated.

Claims (40)

1. A coordination device, comprising:

one or more acoustic transducers configured to measure sound in an environment;

one or more nodes configured to communicatively couple to one or more antennas;

an interface circuit communicatively coupled to the one or more nodes, wherein the coordination device is configured to:

measure the sound using the one or more acoustic transducers associated with electronic devices, wherein the sound corresponds to audio content;

automatically and dynamically aggregate electronic devices into two or more subsets based on the measured sound, wherein the aggregation is based at least in part on sound intensity and acoustic delay, so that proximate speakers are aggregated together;

determine playback timing information for the subsets, wherein the playback timing information specifies playback times when the electronic devices in a given subset are to playback the audio content, and wherein the playback timing information in the given subset is based at least in part on transmit times at the electronic devices and based on clocks in the electronic devices included in frames and receive times at the coordination device and based on a clock in the coordination device of the frames that are communicated between the electronic devices in the given subset and the coordination device; and

transmit, to the one or more nodes, one or more additional frames that include the audio content and playback timing information for the electronic devices, wherein the playback times of the electronic devices in at least the given subset have a temporal relationship so that the playback of the audio content by the electronic devices in the given subset is coordinated.

2. The coordination device of claim 1 , wherein the different subsets are located in different rooms.

3. The coordination device of claim 1 , wherein at least one of the subsets playback different audio content than a remainder of the subsets.

4. The coordination device of claim 3 , wherein the aggregation of the electronic devices into the two or more subsets is further based on the different audio content.

5. The coordination device of claim 1 , wherein the aggregation of the electronic devices into the two or more subsets is based on an acoustic delay of the measured sound.

6. The coordination device of claim 1 , wherein the coordination device is further configured to calculate an estimated location of a listener or another electronic device associated with the listener relative to the electronic devices; and

wherein the aggregation of the electronic devices into the two or more subsets is based on the estimated location of the listener.

7. The coordination device of claim 1 , wherein the aggregation of the electronic devices into the two or more subsets is based on a desired acoustic characteristic in the environment.

8. The coordination device of claim 1 , wherein the coordination device is further configured to modify the measured sound based on an acoustic transfer function of the environment in at least a band of frequencies.

9. The coordination device of claim 1 , wherein the coordination device is further configured to determine playback volumes for the subsets that are used when the subsets playback the audio content; and

wherein the one or more additional frames further include information that specifies the playback volumes.

10. The coordination device of claim 9 , wherein a playback volume for at least one of the subsets is different than the playback volumes of a remainder of the subsets.

11. The coordination device of claim 9 , wherein the playback volumes reduce acoustic cross-talk among the two or more subsets.

12. A non-transitory computer-readable storage medium for use with a coordination device, the computer-readable storage medium storing a program module that, when executed by the coordination device, causes the coordination device to aggregate electronic devices by carrying out one or more operations that comprise:

measuring, using one or more acoustic transducers, sound in an environment associated with the electronic devices, wherein the sound corresponds to audio content;

automatically and dynamically aggregating the electronic devices into two or more subsets based on the measured sound, wherein the aggregation is based at least in part on sound intensity and acoustic delay, so that proximate speakers are aggregated together;

determining playback timing information for the subsets, wherein the playback timing information specifies playback times when the electronic devices in a given subset are to playback the audio content, and wherein the playback timing information in the given subset is based at least in part on transmit times at the electronic devices and based on clocks in the electronic devices included in frames and receive times at the coordination device and based on a clock in the coordination device of the frames that are communicated between the electronic devices in the given subset and the coordination device; and

transmitting, to one or more nodes in the coordination device that are communicatively coupled to one or more antennas, one or more additional frames that include the audio content and playback timing information for the electronic devices, wherein the playback times of the electronic devices in at least the given subset have a temporal relationship so that the playback of the audio content by the electronic devices in the given subset is coordinated.

13. The computer-readable storage medium of claim 12 , wherein at least one of the subsets playback different audio content than a remainder of the subsets.

14. The computer-readable storage medium of claim 13 , wherein the aggregation of the electronic devices into the two or more subsets is further based on the different audio content.

15. The computer-readable storage medium of claim 12 , wherein the aggregation of the electronic devices into the two or more subsets is based on an acoustic delay of the measured sound.

16. The computer-readable storage medium of claim 12 , wherein the one or more operations comprise calculating an estimated location of a listener or another electronic device associated with the listener relative to the electronic devices; and

wherein the aggregation of the electronic devices into the two or more subsets is based on the estimated location of the listener.

17. The computer-readable storage medium of claim 12 , wherein the aggregation of the electronic devices into the two or more subsets is based on a desired acoustic characteristic in the environment.

18. The computer-readable storage medium of claim 12 , wherein the one or more operations comprise modifying the measured sound based on an acoustic transfer function of the environment in at least a band of frequencies.

19. The computer-readable storage medium of claim 12 , wherein the one or more operations comprise determining playback volumes for the subsets that are used when the subsets playback the audio content; and

wherein the one or more additional frames further include information that specifies the playback volumes.

20. A method for aggregating electronic devices, wherein the method comprises:

by a coordination device:

measuring, using one or more acoustic transducers, sound in an environment associated with the electronic devices, wherein the sound corresponds to audio content;

automatically and dynamically aggregating the electronic devices into two or more subsets based on the measured sound, wherein the aggregation is based at least in part on sound intensity and acoustic delay, so that proximate speakers are aggregated together;

determining playback timing information for the subsets, wherein the playback timing information specifies playback times when the electronic devices in a given subset are to playback the audio content, and wherein the playback timing information in the given subset is based at least in part on transmit times at the electronic devices and based on clocks in the electronic devices included in frames and receive times at the coordination device and based on a clock in the coordination device of the frames that are communicated between the electronic devices in the given subset and the coordination device; and

transmitting, to one or more nodes in the coordination device that are communicatively coupled to one or more antennas, one or more additional frames that include the audio content and playback timing information for the electronic devices, wherein the playback times of the electronic devices in at least the given subset have a temporal relationship so that the playback of the audio content by the electronic devices in the given subset is coordinated.

Assignments (11)
RELEASE OF SECURITY INTEREST IN PATENT COLLATERAL (REEL/FRAME 057187/0572) Recorded Apr 12, 2022
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: B & W GROUP LTD
Reel/Frame 059988/0738 →
RELEASE OF SECURITY INTEREST IN PATENT COLLATERAL (REEL/FRAME 057187/0613) Recorded Apr 12, 2022
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: B & W GROUP LTD
Reel/Frame 059988/0688 →
ABL PATENT SECURITY AGREEMENT Recorded Aug 16, 2021
From: B & W GROUP LTD
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 057187/0572 →
FIRST LIEN PATENT SECURITY AGREEMENT Recorded Aug 16, 2021
From: B & W GROUP LTD
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 057187/0613 →
RELEASE OF SECURITY INTEREST (SEE DOCUMENT FOR DETAILS) Recorded Dec 16, 2020
From: LUCID TRUSTEE SERVICES LIMITED
To: EVA AUTOMATION, INC.; EVA HOLDING, CORP.; EVA OPERATIONS CORP.
Reel/Frame 054791/0087 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2020
From: LUCID TRUSTEE SERVICES LIMITED, ACTING AS ATTORNEY-IN-FACT FOR EVA AUTOMATION INC., EVA HOLDING CORP. AND EVA OPERATIONS CORP., AND AS SECURITY AGENT
To: B&W GROUP LTD
Reel/Frame 054765/0526 →
RELEASE OF PATENT COLLATERAL SECURITY AND PLEDGE AGREEMENT Recorded Oct 9, 2020
From: LUCID TRUSTEE SERVICES LIMITED
To: EVA AUTOMATION, INC.
Reel/Frame 054288/0568 →
ASSIGNMENT OF PATENT COLLATERAL SECURITY AND PLEDGE AGREEMENT Recorded Oct 1, 2020
From: BANK OF AMERICA, N.A.
To: LUCID TRUSTEE SERVICES LIMITED
Reel/Frame 053968/0953 →
SECURITY INTEREST Recorded Feb 28, 2019
From: EVA AUTOMATION, INC.
To: LUCID TRUSTEE SERVICES LIMITED
Reel/Frame 048473/0646 →
PATENT COLLATERAL SECURITY AND PLEDGE AGREEMENT Recorded Feb 11, 2019
From: EVA AUTOMATION, INC.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 048301/0213 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 11, 2017
From: YU, GAYLORD; STUPP, STEVEN
To: EVA AUTOMATION, INC.
Reel/Frame 044422/0427 →
Continuity (2)
Provisional Application 62433237 · Dec 13, 2016
Related Publication 20180167758A1 · Jun 14, 2018