Monitor mixing system that distributes real-time multichannel audio over a wireless digital network
In an embodiment, a wireless mix monitor system is provided in which the musicians have one or more wearable electronic devices that may receive digital audio over a wireless connection to a computer. The channels monitored by a given user may be mixed on the computer, on the devices, or on a combination of the computer and the devices. The musicians may be freed from a fixed mix monitor, allowing easy movement about the stage or performance space. Audio buffering and upsampling/downsampling may be used to manage temporal discontinuities in the audio stream.
1. A system comprising:
a computer coupled to one or more digital audio devices and configured to receive a digital audio stream from the one or more digital audio devices during a live performance by one or more musicians, wherein the digital audio stream includes digital audio produced by one or more instruments played by the one or more musicians during the live performance, wherein the computer is configured to packetize the digital audio stream for transmission on a wireless network;
one or more portable devices wirelessly coupled to the wireless network and configured to:
receive the packetized digital audio stream from the wireless network;
mix a plurality of channels of the digital audio stream into a playback stream;
play the playback stream for the one or more musicians;
calculate an average delivery rate of audio samples from the digital audio stream;
calculate an average playback rate of audio samples from the playback stream by an internal audio player in a given one of the one or more portable devices;
compare the average delivery rate to the average playback rate to determine an amount of resampling to be performed to maintain synchronization of the playback stream to the digital audio stream; and
resample the playback stream based on the amount of resampling.
2. The system as recited in claim 1 wherein the one or more portable devices are configured to resample by upsampling in response to detecting discontinuities in the samples.
3. The system as recited in claim 1 wherein the one or more portable devices are configured to resample by downsampling in response to detecting lag in playing back the samples.
4. The system as recited in claim 1 wherein the one or more portable devices are configured to transmit messages to the computer reporting audio reception statistics experienced by the one or more portable devices.
5. The system as recited in claim 4 wherein the computer, responsive to the messages, is configured to alter a workload balance between the computer and the one or more portable devices.
6. The system as recited in claim 5 wherein the computer is configured to alter the workload balance on a per-portable-device basis.
7. The system as recited in claim 4 wherein the computer, responsive to the messages, is configured to perform sample rate conversion.
8. The system as recited in claim 7 wherein the computer, responsive to the messages, is configured to perform sample buffer resizing.
9. The system as recited in claim 1 wherein the one or more portable devices are configured to:
monitor a buffer level for a buffer of the digital audio samples; and
determine an additional amount of resampling to perform based on the buffer level to compensate for discontinuities caused by jitter and latency.
10. A non-transitory computer accessible storage medium storing a plurality of instructions which, when executed, cause a device to:
receive a packetized digital audio stream from a wireless network, the packetized digital audio stream comprising multi-channel audio produced by one or more musicians during a live performance, wherein the digital audio stream includes digital audio produced by one or more instruments played by the one or more musicians during the live performance;
mix a plurality of channels of the packetized digital audio stream into a playback stream to generate a monitor mix for a first musician of the one or more musicians during the live performance;
play the monitor mix for the first musician during the live performance;
calculate an average delivery rate of audio samples from the digital audio stream;
calculate an average playback rate of audio samples from the playback stream by an internal audio player in the device;
compare the average delivery rate to the average playback rate to determine an amount of resampling to be performed to maintain synchronization of the playback stream to the digital audio stream; and
resample the playback stream based on the amount of resampling.
11. The non-transitory computer accessible storage medium as recited in claim 10 wherein the resample includes upsampling in response to detecting discontinuities in the samples.
12. The non-transitory computer accessible storage medium as recited in claim 10 wherein the resample includes downsampling in response to detecting lag in playing back the samples.
13. The non-transitory computer accessible storage medium as recited in claim 10 wherein the plurality of instructions which, when executed, cause the device to transmit messages reporting audio reception statistics experienced by the device.
14. The non-transitory computer accessible storage medium as recited in claim 10 wherein the one or more instructions, when executed by the device, cause the device to:
monitor a buffer level for a buffer of the digital audio samples; and
determine an additional amount of resampling to perform based on the buffer level to compensate for discontinuities caused by jitter and latency.