Method and system for low latency high quality music conferencing
View Patent ↗A method and system for real-time, low latency, high quality audio conferencing are disclosed. The system allows delivering low latency during peer to peer transmission of high quality compressed audio streams between remotely located participants. The system provides transmission of audio as well as any audio data with low latency and high quality. The system solves latency problems to enable participants in different locations to stay in synchronization while performing live over the Internet in multiple locations.
1. An audio conferencing system comprising:
a plug-in hardware device coupled with a client computer;
a sound card coupled with the client computer; and
a modified Windows operating system running on the client computer including:
a custom network stack comprising real-time transfer protocol, an adaptive jitter buffer algorithm, a packet loss prevention algorithm, a bandwidth adaptation mechanism utilizing layered coding mechanism, time synchronization with network time protocol and stream synchronization and mixing algorithms;
a custom audio stack comprising a port driver, an audio port driver configured to eliminate the handling of mappings and the need to manipulate audio data in one or more audio streams, a wave miniport driver, a stream mixer, and an adapter driver; and
one or more audio conferencing enhancements.
2. The system of claim 1 wherein said hardware device includes a musical instrument.
3. The system of claim 1 wherein both the custom network stack and the custom audio stack process audio data received over a network in kernel-mode only and never go into user-mode to avoid I/O delays resulting from switching between user-mode and kernel-mode which leads to audio glitches and distortions during high CPU load.
4. The system of claim 3 wherein said audio conferencing enhancements provide musical instruments and voice compression optimization where different compression codecs are used for different musical instruments and voices.
5. The system of claim 3 wherein said audio conferencing enhancements provide musical instruments and voice decompression optimization where different decompression codecs are used for different musical instruments and voices.
6. The system of claim 3 wherein said network stack comprises a real-time transfer protocol with custom extensions for voice and various musical instruments.
7. The system of claim 3 wherein said network stack comprises an adaptive jitter buffer algorithm designed to remove uttering effects from an audio stream.
8. The system of claim 3 wherein said network stack comprises a Reed-Solomon based Forward Error Correction algorithm designed to prevent packet loss.
9. The system of claim 3 wherein said network stack comprises a bandwidth adaptation with layered coding mechanism.
10. The system of claim 3 wherein said network stack comprises time synchronization with network time protocol.
11. The system of claim 3 wherein network latency is no more than 50 ms.
12. The system of claim 3 wherein said stream mixer of the custom audio stack utilizes direct kernel streaming technology which allows by passing a generic high-latency audio stack of the Windows operating system for direct drive or communications.
13. The system of claim 3 wherein said audio conferencing enhancements provide voice and rhythm transport optimization.
14. The system of claim 3 wherein said audio conferencing enhancements provide conference topology optimization.
15. The system of claim 3 wherein said audio conferencing enhancements provide musical instruments and voice compression optimization.
16. The system of claim 3 wherein said audio conferencing enhancements provide metronome facilitated high latency audio performance.
17. The system of claim 3 wherein said audio conferencing enhancements provide smart volume management for packet loss concealment.