IP Library Granted Patent US 12,248,725
Granted Patent B2
US 12,248,725 · App. 17/902,486 · Granted Mar 11, 2025

Systems and methods for providing real-time audio and data

Inventors: Michael Einziger (Santa Monica, CA); Ann Marie Simpson (Santa Monica, CA); Darren LaGroe (Westlake Village, CA); Michal Pietras (London, GB)
Assignee: Mixhalo Corp.
G06F3/162G06F3/00H04L65/1059H04L65/65H04L65/70H04L65/762H04L65/764H04L65/765H04L65/80H04W4/02H04W4/06
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,248,725
App. No.
17/902,486
Granted
Mar 11, 2025
Kind
B2
Abstract

A method of delivering audio to one or more client computing devices includes receiving, by an audio server computing device, a live audio signal starting at a first time; processing, by the audio server computing device, the live audio signal, thereby creating a data representation of the live audio signal; transmitting, by the audio server computing device, via a wireless network in electronic communication with the audio server computing device, the data representation of the live audio signal to the one or more client computing devices; interpreting, by the one or more client computing devices, the data representation of the live audio signal, thereby producing an interpreted audio signal; and providing, by the one or more client computing devices, the interpreted audio signal to a user listening device starting at a second time. A latency between the first time and the second time is less than 100 milliseconds.

Claims (34)

1. A method of minimizing total latency of a streaming audio signal produced by an audio system, the method comprising streaming a received live audio signal over a wireless network between an audio server and a user listening device coupled to a smartphone and individually minimizing a latency of each of the following audio system components that are in a path of streaming of the received live audio signal between the audio server and the user listening device:

the audio server's audio hardware interface;

the audio server's audio hardware interface driver;

an audio server application;

the audio server's wireless network stack including a wireless network hardware interface driver;

the audio server's wireless network hardware interface;

an Ethernet wireless network;

the wireless network's access port;

the wireless network;

the smartphone's hardware wireless network interface;

the smartphone's operating system wireless network stack including a hardware wireless network interface driver;

The smartphone's client application;

the smartphone's operating system including an audio hardware interface driver; and

the smartphone's audio hardware interface coupled to the user listening device,

wherein a sum of latencies of each of the above audio system components from a first time of receiving the live audio signal by the audio server's audio hardware interface to a second time of outputting the live audio signal to the user listening device by the smartphone's audio hardware interface is less than 100 milliseconds.

2. The method of claim 1 wherein the sum of latencies is less than 50 milliseconds.

3. The method of claim 1 wherein the sum of latencies is less than 20 milliseconds.

4. The method of claim 1 wherein the audio server's audio hardware interface comprises a sound card.

5. The method of claim 1 wherein the audio server application comprises an audio processing software.

6. The method of claim 5 wherein the audio processing software is configured to adjust at least one of bit depth, sample rate, buffer settings, or routing settings.

7. The method of claim 5 wherein the audio processing software is configured to packetize and transmit audio data over Real-time Transport Protocol.

8. The method of claim 7 wherein the audio processing software is configured to packetize and transmit L16PCM audio data over Real-time Transport Protocol.

9. The method of claim 1 wherein the Ethernet wireless network is capable of at least 10 Mb/s communication.

10. The method of claim 9 wherein the Ethernet wireless network is capable of at least 100 Mb/s communication.

11. The method of claim 10 wherein the Ethernet wireless network is capable of at least 1000 Mb/s communication.

12. The method of claim 1 wherein the wireless network is capable of supporting at least one of unicast, multicast, or broadcast.

13. The method of claim 1 wherein minimizing the latency of the wireless network comprises enabling multicasting.

14. The method of claim 1 wherein minimizing the latency of the wireless network comprises disabling multicast IGMP snooping.

15. The method of claim 1 wherein minimizing the latency of the wireless network comprises setting a beacon time.

16. The method of claim 1 wherein minimizing the latency of the wireless network comprises setting a DTIM interval.

17. The method of claim 1 wherein minimizing the latency of the wireless network comprises setting a multicast rate.

18. The method of claim 1 wherein minimizing the latency of the wireless network comprises reprioritizing multicast traffic.

19. The method of claim 1 wherein minimizing the latency of the wireless network comprises ignoring nonessential traffic.

20. The method of claim 1 wherein minimizing the latency of the wireless network comprises disabling client PSM.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 15, 2023
From: EINZIGER, MICHAEL; SIMPSON, ANN MARIE; LAGROE, DARREN; PIETRAS, MICHAL
To: MIXHALO CORP.
Reel/Frame 065565/0969 →
Continuity (4)
Division 15978726 · May 14, 2018
Provisional Application 62639346 · Mar 6, 2018
Provisional Application 62506481 · May 15, 2017
Related Publication 20230075582A1 · Mar 9, 2023
References Cited (130)
US 4593404A · Bolin · 1986 [cited by applicant]
US 5131051A · Kishinaga et al. · 1992 [cited by applicant]
US 5619582A · Ottman et al. · 1997 [cited by applicant]
US 5822440A · Ottman et al. · 1998 [cited by applicant]
US RE38405E · Clair, Jr. et al. · 2004 [cited by applicant]
US 6680906B1 · Nguyen · 2004 [cited by applicant]
US 7995770B1 · Simon · 2011 [cited by applicant]
US 8290174B1 · Simon · 2012 [cited by applicant]
US 8379874B1 · Simon · 2013 [cited by applicant]
US 8577053B1 · Simon · 2013 [cited by applicant]
US 8588432B1 · Simon · 2013 [cited by applicant]
US 8879455B1 · Stephenson et al. · 2014 [cited by applicant]
US 8938078B2 · Meyer · 2015 [cited by applicant]
US 9137823B1 · Liu et al. · 2015 [cited by applicant]
US 9154318B2 · Witt et al. · 2015 [cited by applicant]
US 9232471B2 · Choi et al. · 2016 [cited by applicant]
US 9247337B2 · Foudhaili et al. · 2016 [cited by applicant]
US 9398394B2 · Bongiovi · 2016 [cited by applicant]
US 9635482B2 · Menke · 2017 [cited by applicant]
US 9635484B2 · Kuhr et al. · 2017 [cited by applicant]
US 9716946B2 · Goshen et al. · 2017 [cited by applicant]
US 9788108B2 · Goshen et al. · 2017 [cited by applicant]
US 9841942B2 · Tull · 2017 [cited by applicant]
US 9864573B2 · Tull · 2018 [cited by applicant]
US 9894445B2 · Feldt · 2018 [cited by applicant]
US 9916835B2 · Frey et al. · 2018 [cited by applicant]
US 9918179B2 · Kuhr et al. · 2018 [cited by applicant]
US 9930462B2 · Goshen et al. · 2018 [cited by applicant]
US 9942675B2 · Tull · 2018 [cited by applicant]
US 10001968B1 · Slick · 2018 [cited by applicant]
US 10003901B1 · Slick · 2018 [cited by applicant]
US 10104147B2 · Schmidt et al. · 2018 [cited by applicant]
US 10236031B1 · Gurijala · 2019 [cited by examiner]
US 10257074B1 · Singh · 2019 [cited by examiner]
US 10299059B1 · Slick · 2019 [cited by applicant]
US 10362379B2 · Georgi et al. · 2019 [cited by applicant]
US 10375134B2 · Cauduro Dias de Paiva et al. · 2019 [cited by applicant]
US 10454982B1 · Slick · 2019 [cited by examiner]
US 10609464B2 · Georgi et al. · 2020 [cited by applicant]
US 10616303B2 · Christiansen et al. · 2020 [cited by applicant]
US 10686897B2 · Werner et al. · 2020 [cited by applicant]
US 10827253B2 · Boetcher et al. · 2020 [cited by applicant]
US 20020013833A1 · Wyatt et al. · 2002 [cited by applicant]
US 20040103282A1 · Meier · 2004 [cited by applicant]
US 20050086289A1 · Rosenberg · 2005 [cited by applicant]
US 20070140510A1 · Redmann · 2007 [cited by applicant]
US 20070155312A1 · Goldberg · 2007 [cited by applicant]
US 20070223675A1 · Surin · 2007 [cited by examiner]
US 20080189750A1 · Yoon · 2008 [cited by applicant]
US 20080201424A1 · Darcie · 2008 [cited by applicant]
US 20090010191A1 · Wentink · 2009 [cited by applicant]
US 20090113022A1 · Quoc · 2009 [cited by examiner]
US 20090240998A1 · Nikkila et al. · 2009 [cited by applicant]
US 20090276686A1 · Liu et al. · 2009 [cited by applicant]
US 20100110962A1 · Igarashi et al. · 2010 [cited by applicant]
US 20110055444A1 · Henriksson · 2011 [cited by applicant]
US 20110185286A1 · Moyers · 2011 [cited by applicant]
US 20120079067A1 · Stout · 2012 [cited by applicant]
US 20130022131A1 · Ocon et al. · 2013 [cited by applicant]
US 20140140209A1 · Shihada et al. · 2014 [cited by applicant]
US 20140181179A1 · Clark · 2014 [cited by applicant]
US 20140297799A1 · Gordon · 2014 [cited by examiner]
US 20140328485A1 · Saulters · 2014 [cited by applicant]
US 20150180748A1 · Ding · 2015 [cited by applicant]
US 20150254056A1 · Walker et al. · 2015 [cited by applicant]
US 20150256473A1 · Walker · 2015 [cited by applicant]
US 20150296502A1 · Gerhardt et al. · 2015 [cited by applicant]
US 20150317197A1 · Blair · 2015 [cited by applicant]
US 20150319583A1 · McIntyre · 2015 [cited by applicant]
US 20150341758A1 · Aggarwal · 2015 [cited by applicant]
US 20160037176A1 · Chari · 2016 [cited by applicant]
US 20160057522A1 · Choisel · 2016 [cited by applicant]
US 20160100362A1 · Palanisamy et al. · 2016 [cited by applicant]
US 20160255131A1 · Bulava et al. · 2016 [cited by applicant]
US 20160278007A1 · Gokhale · 2016 [cited by applicant]
US 20160286260A1 · Lawrence · 2016 [cited by examiner]
US 20160360256A1 · Van Os · 2016 [cited by applicant]
US 20170142535A1 · Aggarwal · 2017 [cited by examiner]
US 20170201940A1 · Koratekere Honnappa · 2017 [cited by examiner]
US 20170235544A1 · Nyegaard et al. · 2017 [cited by applicant]
US 20180020287A1 · Goshen et al. · 2018 [cited by applicant]
US 20180069662A1 · Sugaya et al. · 2018 [cited by applicant]
US 20180234742A1 · Abrams · 2018 [cited by applicant]
US 20180249277A1 · Voss et al. · 2018 [cited by applicant]
US 20180270139A1 · Singh · 2018 [cited by applicant]
US 20190173938A1 · Tull · 2019 [cited by applicant]
US 20190179597A1 · Tull et al. · 2019 [cited by applicant]
US 20190182557A1 · Tull · 2019 [cited by applicant]
US 20190349608A1 · Pollet · 2019 [cited by applicant]
US 20190356718A1 · Haustein et al. · 2019 [cited by applicant]
US 20200014742A1 · Pollet · 2020 [cited by applicant]
GB 2552016A · 2018 [cited by examiner]
JP 2009005114A · 2009 [cited by applicant]
JP 4407369B2 · 2010 [cited by applicant]
JP 2010068111A · 2010 [cited by applicant]
JP 2012212142A · 2012 [cited by applicant]
WO WO2006110975A1 · 2006 [cited by applicant]
WO WO2014154822A1 · 2014 [cited by applicant]
Ableton “how to Reduce Latency” (Year: 2017). [cited by examiner]
Michal, “Latency and Recording Guitars”, Apr. 27, 2016 (Year: 2016). [cited by examiner]
Jakubisin , “Real-Time Audio Transceiver Utilizing 802.11b Wireless Technology” (Year: 2007). [cited by examiner]
D. Shapira and S. Dubnov, “DistributeDJ: A Mobile Group Music Making Toolkit,” 2013 IEEE Seventh International Conference on Semantic Computing, Irvine, CA, USA, 2013, pp. 334-338 (Year: 2013). [cited by examiner]
Agron, P., & Févotte, C. (2022). Neural content-aware collaborative filtering for cold-start music recommendation. Data Mining and Knowledge Discovery, , 1971-2005 (Year: 2005). [cited by examiner]
G. Vlachakis, N. Karadimitriou and D. Akoumianakis, “Using a dedicated toolkit and the cloud to coordinate shared music representations,” IISA 2014, The 5th International Conference on Information, Intelligence, Systems… [cited by examiner]
D. Akoumianakis, G. Ktistakis, G. Vlachakis, P. Zervas and C. Alexandraki, “Collaborative music making as ‘remediated’ practice,” 2013 18th International Conference on Digital Signal Processing (DSP), Fira, Greece, 2013… [cited by examiner]
Cisco Systems “IP Multicast Network Management”. [cited by applicant]
Cutietta “The Internet for Ensemble Performance” 2004. [cited by applicant]
Denon “Wireless, AP or Client Isolation” 2016. [cited by applicant]
Discover New Worlds With MobileConnect, 2014. [cited by applicant]
El-Shimy et al “EmbodiNet: Enriching Distributed Musical Collaboration Through Embodied Interactions” 2015. [cited by applicant]
Fitzek et al “Robust Header Compression (ROHC) Performance for Multimedia Transmission Over 3G/4G Wireless Networks” Wireless Pers Commun vol. 32, pp. 32-41, 2005. [cited by applicant]
Fong et al “Online eLearning System for Live Band Practicing” International Symposium on Educational Technology, pp. 69-72, 2016. [cited by applicant]
Gettelman “Yes Mines Cutting Edge of Sound” Orlando Sentinel, 1994. [cited by applicant]
Giuli et al “Orchestral: A Distributed Platform for Virtual Musical Groups and Music Distance Learning Over the Internet in Java/Sup TM/Technology” Proceedings IEEE International Conference on Multimedia Computing and S… [cited by applicant]
Kening “Yes Answers Critics Who Thing They're Ready for Scraphead” Chicago Tribune, 1994. [cited by applicant]
Lee “What is the Difference Between UDP and TCP Internet Protocols?” Reliability Handbook, 2019. [cited by applicant]
Listen Technologies Press Release, 2015. [cited by applicant]
Listen Technologies Product Overview, 2016. [cited by applicant]
Multicast With Wireless LAN Controllers (WLCs) and Lightweight Access Points (LAPs) Configuration Example. [cited by applicant]
Pei et al “Why It Takes So Long to Connect to a WiFi Access Point” IEEE Conference on Computer Communications, pp. 1-9, 2017. [cited by applicant]
Sawchuk “From Remote Media Immersion to Distributed Immersive Performance” 2003. [cited by applicant]
Sui et al “Characterizing and Improving WiFi Latency in Large-Scale Operational Networks” pp. 347-360. [cited by applicant]
Third-Party Submission Under 37 CFR 1.290 in U.S. Appl. No. 15/978,754, 2019. [cited by applicant]
Vlachakis et al “Using a Dedicated Toolkit and the Cloud to Coordinate Shared Music Representations” The 5 [cited by applicant]
VLC Media Player, 2001. [cited by applicant]
Wikapedia Encyclopedia, Talk (Yes Album). [cited by applicant]
Apple App Store product preview URL:https://apps.apple.com/us/app/listenwifi-personal-listening/id782256496, Apr. 23, 2014—Version 1.0.7, released; Jan. 26, 2017—Version 2.3.0 released, 1 page. [cited by applicant]
J. Kleimola, “Latency Issues in Distributed Musical Performance,” Helsinki Univ. of Technology, Telecommunications Software and Multimedia Laboratory, T-111.5080 Seminar on content creation, Fall 2006, dated Dec. 20, 20… [cited by applicant]
International Search Report and Written Opinion of the International Searching Authority in International Application No. PCT/US2018/032532, dated Oct. 9, 2018, 18 pages. [cited by applicant]
International Search Report and Written Opinion of the International Searching Authority in International Application No. PCT/US2018/032535, dated Aug. 29, 2018, 13 pages. [cited by applicant]