IP Library Granted Patent US 10,649,722
Granted Patent B2
US 10,649,722 · App. 16/266,100 · Granted May 12, 2020

Schedule-based coordination of audio sources

Inventors: Gaylord Yu (San Francisco, CA); Steven Stupp (Foster City, CA)
Assignee: EVA Automation, Inc.
G06F3/165G06F1/12G11B27/10H04L7/02H04L65/60H04N21/4305H04N21/43615H04N21/8106H04N21/8547H04R27/00H04S7/301H04S7/307H04W4/023H04W56/005H04L29/06027H04R3/12H04R2227/005H04R2420/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,649,722
App. No.
16/266,100
Granted
May 12, 2020
Kind
B2
Abstract

An audio/video (A/V) hub that coordinates playback of audio content is described. In particular, the A/V hub may calculate current time offsets between clocks in electronic devices and a clock in the A/V hub based on differences between receive times when frames are received from electronic devices and expected transmit times of the frames. For example, the expected transmit times may be based on coordination of clocks in the electronic devices and a clock in the A/V hub at a previous time and a predefined transmit schedule of the frames. Then, the A/V hub may transmit, to the electronic devices, one or more frames that include audio content and playback timing information, which may specify playback times when the electronic devices are to playback the audio content based on the current time offsets.

Claims (57)

1. A coordination device, comprising:

one or more nodes configured to communicatively couple to one or more antennas;

an interface circuit communicatively coupled to the one or more nodes, wherein the coordination device is configured to:

receive, from the one or more nodes, input frames associated with electronic devices;

store receive times when the input frames were received, wherein the receive times are based on a clock in the coordination device and are determined in a physical layer in the coordination device;

calculate current time offsets between clocks in the electronic devices and the clock in the coordination device based on the receive times and expected transmit times of the input frames, wherein the expected transmit times are based on coordination of the clocks in the electronic devices and the clock in the coordination device at a previous time and a predefined transmit schedule of the input frames; and

transmit, via the one or more nodes, one or more output frames that comprise audio content and playback timing information intended for the electronic devices, wherein the playback timing information specifies playback times when the electronic devices are to playback the audio content based on the current time offsets,

wherein the playback times of the electronic devices have a temporal relationship so that the playback of the audio content by the electronic devices is coordinated, and

wherein the coordination device is configured to receive the input frames and transmit the one or more output frames using wireless communication.

2. The coordination device of claim 1 , wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times.

3. The coordination device of claim 2 , wherein the different playback times are based on acoustic characterization of an environment.

4. The coordination device of claim 2 , wherein the different playback times are based on a desired acoustic characteristic in an environment.

5. The coordination device of claim 2 , wherein the electronic devices are located at vector distances from the coordination device;

wherein the interface circuit is configured to determine magnitudes of the vector distances based on the expected transmit times of the input frames and the receive times using wireless ranging, and is configured to determine angles of the vector distances based on the angle of arrival of wireless signals associated with the input frames; and

wherein the different playback times are based on the determined vector distances.

6. The coordination device of claim 2 , wherein the different playback times are based on an estimated location of a listener relative to the electronic devices.

7. The coordination device of claim 6 , wherein the interface circuit is configured to:

communicate, to or from the one or more nodes, second frames associated with another electronic device; and

calculate the estimated location of the listener based on the communication with the other electronic device.

8. The coordination device of claim 6 , wherein the coordination device further comprises an acoustic transducer configured to perform sound measurements of an environment; and

wherein the coordination device is configured to calculate the estimated location of the listener based on the sound measurements.

9. The coordination device of claim 6 , wherein the interface circuit is configured to communicate, to or from the one or more nodes, third frames associated with other electronic devices in the environment and to receive, from the one or more nodes, additional sound measurements, associated with the other electronic devices, of the environment; and

wherein the coordination device is configured to calculate the estimated location of the listener based on the additional sound measurements.

10. The coordination device of claim 6 , wherein the interface circuit is configured to:

perform time-of-flight measurements; and

calculate the estimated location of the listener based on the time-of-flight measurements.

11. The coordination device of claim 1 , wherein the coordination of the clocks in the electronic devices and the clock in the coordination device occurred during an initialization mode of operation.

12. The coordination device of claim 1 , wherein the current time offsets are further based on models of clock drift in the electronic devices.

13. A non-transitory computer-readable storage medium for use with a coordination device, the computer-readable storage medium storing a program module that, when executed by the coordination device, causes the coordination device to coordinate playback of audio content by carrying out one or more operations that comprise:

receiving, from one or more nodes in the coordination device that are communicatively coupled to one or more antennas, input frames associated with electronic devices;

storing receive times when the input frames were received, wherein the receive times are based on a clock in the coordination device and are determined in a physical layer in the coordination device;

calculating current time offsets between clocks in the electronic devices and the clock in the coordination device based on the receive times and expected transmit times of the input frames, wherein the expected transmit times are based on coordination of the clocks in the electronic devices and the clock in the coordination device at a previous time and a predefined transmit schedule of the input frames; and

transmitting, via the one or more nodes, one or more output frames that comprise the audio content and playback timing information intended for the electronic devices, wherein the playback timing information specifies playback times when the electronic devices are to playback the audio content based on the current time offsets,

wherein the playback times of the electronic devices have a temporal relationship so that the playback of the audio content by the electronic devices is coordinated, and

wherein the coordination device is configured to receive the input frames and transmit the one or more output frames using wireless communication.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times; and

wherein the different playback times are based on acoustic characterization of an environment.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times; and

wherein the different playback times are based on a desired acoustic characteristic in the environment.

16. The non-transitory computer-readable storage medium of claim 14 , wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times;

wherein the electronic devices are located at vector distances from the coordination device;

wherein the one or more operations comprise determining magnitudes of the vector distances based on the expected transmit times of the input frames and the receive times using wireless ranging, and determining angles of the vector distances based on the angle of arrival of wireless signals associated with the input frames; and

wherein the different playback times are based on the determined vector distances.

17. The non-transitory computer-readable storage medium of claim 14 , wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times; and

wherein the different playback times are based on an estimated location of a listener relative to the electronic devices.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more operations comprise:

performing time-of-flight measurements; and

calculating the estimated location of the listener based on the time-of-flight measurements.

19. The non-transitory computer-readable storage medium of claim 13 , wherein the current time offsets are further based on models of clock drift in the electronic devices.

20. A method for coordinating playback of audio content, wherein the method comprises:

by a coordination device:

receiving, from one or more nodes in the coordination device that are communicatively coupled to one or more antennas, input frames associated with electronic devices;

storing receive times when the input frames were received, wherein the receive times are based on a clock in the coordination device and are determined in a physical layer in the coordination device;

calculating current time offsets between clocks in the electronic devices and the clock in the coordination device based on the receive times and expected transmit times of the input frames, wherein the expected transmit times are based on coordination of the clocks in the electronic devices and the clock in the coordination device at a previous time and a predefined transmit schedule of the input frames; and

transmitting, via the one or more nodes, one or more output frames that comprise audio content and playback timing information intended for the electronic devices, wherein the playback timing information specifies playback times when the electronic devices are to playback the audio content based on the current time offsets,

wherein the playback times of the electronic devices have a temporal relationship so that the playback of the audio content by the electronic devices is coordinated, and

wherein the coordination device receives the input frames and transmits the one or more output frames using wireless communication.

Assignments (8)
RELEASE OF SECURITY INTEREST IN PATENT COLLATERAL (REEL/FRAME 057187/0613) Recorded Apr 12, 2022
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: B & W GROUP LTD
Reel/Frame 059988/0688 →
RELEASE OF SECURITY INTEREST IN PATENT COLLATERAL (REEL/FRAME 057187/0572) Recorded Apr 12, 2022
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: B & W GROUP LTD
Reel/Frame 059988/0738 →
ABL PATENT SECURITY AGREEMENT Recorded Aug 16, 2021
From: B & W GROUP LTD
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 057187/0572 →
FIRST LIEN PATENT SECURITY AGREEMENT Recorded Aug 16, 2021
From: B & W GROUP LTD
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 057187/0613 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2021
From: DEI SALES, INC.
To: B&W GROUP LTD
Reel/Frame 055174/0024 →
RELEASE OF SECURITY INTEREST Recorded Jan 29, 2021
From: LUCID TRUSTEE SERVICES LIMITED
To: EVA AUTOMATION, INC.; EVA HOLDING, CORP.; EVA OPERATIONS CORP.
Reel/Frame 055174/0064 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2021
From: EVA AUTOMATION, INC.; EVA HOLDING, CORP.; EVA OPERATIONS CORP.
To: DEI SALES, INC.
Reel/Frame 055222/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 3, 2019
From: YU, GAYLORD; STUPP, STEVEN
To: EVA AUTOMATION, INC.
Reel/Frame 048230/0716 →
Continuity (3)
Continuation 15678048 · Aug 15, 2017
Provisional Application 62433237 · Dec 13, 2016
Related Publication 20190171411A1 · Jun 6, 2019
Cited By (1)
US 12,720,267