IP Library Granted Patent US 12,248,726
Granted Patent B2
US 12,248,726 · App. 18/121,346 · Granted Mar 11, 2025

Systems and methods for providing real-time audio and data

Inventors: Michael Einziger (Santa Monica, CA); Ann Marie Simpson (Santa Monica, CA); Nathanial Pollack (San Francisco, CA); Vikram Singh (Los Altos Hills, CA)
Assignee: Mixhalo Corp.
G06F3/162G06F3/00H04L65/1059H04L65/65H04L65/70H04L65/762H04L65/764H04L65/765H04L65/80H04W4/02H04W4/06
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,248,726
App. No.
18/121,346
Granted
Mar 11, 2025
Kind
B2
Abstract

A computerized method of delivering data to one or more client computing devices includes receiving, by a server computing device, a data stream starting at a first time; processing, by the server computing device, the data stream, thereby creating a processed data stream; transmitting, by the server computing device, via a wireless network in electronic communication with the server computing device, the processed data stream to the one or more client computing devices; and interpreting, by an application installed on the one or more client computing devices, the processed data stream, thereby recovering the data stream for use by the one or more client computing devices. A latency between the first time and the second time is less than 100 milliseconds.

Claims (34)

1. A method of minimizing total latency of streaming a live data stream produced by a streaming data system while preserving continuity of the live data stream, the method comprising streaming a received live data stream over a wireless network between a server and a user listening device coupled to a smartphone and individually minimizing a latency of each of the following streaming data system components that are in a path of streaming of the received live data stream between the server and the user listening device:

the server's hardware interface;

the server's hardware interface driver;

a server application;

the server's network stack including a network hardware interface driver;

the server's network hardware interface;

an Ethernet network;

the wireless network's access port;

the wireless network;

the smartphone's hardware wireless network interface;

the smartphone's operating system network stack including a hardware wireless network interface driver;

a smartphone client application;

the smartphone's operating system; and

the smartphone's hardware interface coupled to the user listening device,

wherein a sum of latencies of each of the above streaming data system components from a first time of receiving the live data stream by the server's hardware interface to a second time of outputting the live data stream to the user listening device by the smartphone's hardware interface is less than 100 milliseconds.

2. The method of claim 1 , wherein the sum of latencies is less than 50 milliseconds.

3. The method of claim 1 , wherein the sum of latencies is less than 20 milliseconds.

4. The method of claim 1 wherein the server hardware interface comprises a sound card.

5. The method of claim 1 wherein the server application comprises an audio processing software.

6. The method of claim 5 wherein the audio processing software is configured to adjust at least one of bit depth, sample rate, buffer settings, or routing settings.

7. The method of claim 5 wherein the audio processing software is configured to packetize and transmit audio data over Real-time Transport Protocol.

8. The method of claim 7 wherein the audio processing software is configured to packetize and transmit L16PCM audio data over Real-time Transport Protocol.

9. The method of claim 1 wherein the Ethernet network is capable of at least 10 Mb/s communication.

10. The method of claim 9 wherein the Ethernet network is capable of at least 100 Mb/s communication.

11. The method of claim 10 wherein the Ethernet network is capable of at least 1000 Mb/s communication.

12. The method of claim 1 wherein the wireless network is capable of supporting at least one of unicast, multicast, or broadcast.

13. The method of claim 1 wherein minimizing the latency of the wireless network comprises enabling multicasting.

14. The method of claim 1 wherein minimizing the latency of the wireless network comprises disabling multicast IGMP snooping.

15. The method of claim 1 wherein minimizing the latency of the wireless network comprises setting a beacon time.

16. The method of claim 1 wherein minimizing the latency of the wireless network comprises setting a DTIM interval.

17. The method of claim 1 wherein minimizing the latency of the wireless network comprises setting a multicast rate.

18. The method of claim 1 wherein minimizing the latency of the wireless network comprises reprioritizing multicast traffic.

19. The method of claim 1 wherein minimizing the latency of the wireless network comprises ignoring nonessential traffic.

20. The method of claim 1 wherein minimizing the latency of the wireless network comprises disabling client PSM.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 15, 2023
From: EINZIGER, MICHAEL; SIMPSON, ANN MARIE; POLLACK, NATHANIAL; SINGH, VIKRAM
To: MIXHALO CORP.
Reel/Frame 065566/0303 →
Continuity (4)
Division 15978754 · May 14, 2018
Provisional Application 62639346 · Mar 6, 2018
Provisional Application 62506481 · May 15, 2017
Related Publication 20230297319A1 · Sep 21, 2023
References Cited (121)
US 4593404A · Bolin · 1986 [cited by applicant]
US 5131051A · Kishinaga et al. · 1992 [cited by applicant]
US 5619582A · Oltman et al. · 1997 [cited by applicant]
US 5822440A · Oltman et al. · 1998 [cited by applicant]
US RE38405E · Clair, Jr. et al. · 2004 [cited by applicant]
US 6680906B1 · Nguyen · 2004 [cited by applicant]
US 7995770B1 · Simon · 2011 [cited by applicant]
US 8290174B1 · Simon · 2012 [cited by applicant]
US 8379874B1 · Simon · 2013 [cited by applicant]
US 8577053B1 · Simon · 2013 [cited by applicant]
US 8588432B1 · Simon · 2013 [cited by applicant]
US 8938078B2 · Meyer · 2015 [cited by applicant]
US 9137823B1 · Liu et al. · 2015 [cited by applicant]
US 9247337B2 · Foudhaili et al. · 2016 [cited by applicant]
US 9398394B2 · Bongiovi et al. · 2016 [cited by applicant]
US 9635482B2 · Menke · 2017 [cited by applicant]
US 9635484B2 · Kuhr et al. · 2017 [cited by applicant]
US 9716946B2 · Goshen et al. · 2017 [cited by applicant]
US 9788108B2 · Goshen et al. · 2017 [cited by applicant]
US 9841942B2 · Tull · 2017 [cited by applicant]
US 9864573B2 · Tull · 2018 [cited by applicant]
US 9894445B2 · Feldt · 2018 [cited by applicant]
US 9916835B2 · Frey et al. · 2018 [cited by applicant]
US 9918179B2 · Kuhr et al. · 2018 [cited by applicant]
US 9930462B2 · Goshen et al. · 2018 [cited by applicant]
US 9942675B2 · Tull · 2018 [cited by applicant]
US 10001968B1 · Slick · 2018 [cited by applicant]
US 10003901B1 · Slick · 2018 [cited by applicant]
US 10104147B2 · Schmidt et al. · 2018 [cited by applicant]
US 10236031B1 · Gurijala · 2019 [cited by examiner]
US 10257074B1 · Singh · 2019 [cited by examiner]
US 10299059B1 · Slick · 2019 [cited by applicant]
US 10362379B2 · Georgi et al. · 2019 [cited by applicant]
US 10375134B2 · Cauduro Dias de Paiva et al. · 2019 [cited by applicant]
US 10609464B2 · Georgi et al. · 2020 [cited by applicant]
US 10616303B2 · Christiansen et al. · 2020 [cited by applicant]
US 10686897B2 · Werner et al. · 2020 [cited by applicant]
US 10827253B2 · Boetcher et al. · 2020 [cited by applicant]
US 20020013833A1 · Wyatt et al. · 2002 [cited by applicant]
US 20040103282A1 · Meier et al. · 2004 [cited by applicant]
US 20050086289A1 · Rosenberg et al. · 2005 [cited by applicant]
US 20070223675A1 · Surin · 2007 [cited by examiner]
US 20080049703A1 · Kneckt et al. · 2008 [cited by applicant]
US 20080189750A1 · Yoon et al. · 2008 [cited by applicant]
US 20080201424A1 · Darcie · 2008 [cited by applicant]
US 20090010191A1 · Wentink · 2009 [cited by applicant]
US 20090113022A1 · Quoc · 2009 [cited by examiner]
US 20100110962A1 · Igarashi et al. · 2010 [cited by applicant]
US 20110055444A1 · Henriksson et al. · 2011 [cited by applicant]
US 20110185286A1 · Moyers et al. · 2011 [cited by applicant]
US 20120079067A1 · Stout et al. · 2012 [cited by applicant]
US 20130022131A1 · Ocon et al. · 2013 [cited by applicant]
US 20140140209A1 · Shihada et al. · 2014 [cited by applicant]
US 20140181779A1 · Cao · 2014 [cited by applicant]
US 20140297799A1 · Gordon · 2014 [cited by examiner]
US 20150180748A1 · Ding et al. · 2015 [cited by applicant]
US 20150254056A1 · Walker et al. · 2015 [cited by applicant]
US 20150256473A1 · Walker et al. · 2015 [cited by applicant]
US 20150296502A1 · Gerhardt et al. · 2015 [cited by applicant]
US 20150317197A1 · Blair · 2015 [cited by applicant]
US 20150319583A1 · McIntyre et al. · 2015 [cited by applicant]
US 20150341758A1 · Aggarwal et al. · 2015 [cited by applicant]
US 20160037176A1 · Chari · 2016 [cited by applicant]
US 20160057522A1 · Choisel et al. · 2016 [cited by applicant]
US 20160100362A1 · Palanisamy et al. · 2016 [cited by applicant]
US 20160255131A1 · Bulava et al. · 2016 [cited by applicant]
US 20160278007A1 · Gokhale · 2016 [cited by applicant]
US 20160286260A1 · Lawrence · 2016 [cited by examiner]
US 20160360256A1 · Van Os et al. · 2016 [cited by applicant]
US 20170142535A1 · Aggarwal · 2017 [cited by examiner]
US 20170201940A1 · Koratekere Honnappa · 2017 [cited by examiner]
US 20170235544A1 · Nyegaard et al. · 2017 [cited by applicant]
US 20180020287A1 · Goshen et al. · 2018 [cited by applicant]
US 20180069662A1 · Sugaya et al. · 2018 [cited by applicant]
US 20180234742A1 · Abrams · 2018 [cited by applicant]
US 20180249277A1 · Voss et al. · 2018 [cited by applicant]
US 20180270139A1 · Singh · 2018 [cited by applicant]
US 20190173938A1 · Tull · 2019 [cited by applicant]
US 20190179597A1 · Tull et al. · 2019 [cited by applicant]
US 20190182557A1 · Tull · 2019 [cited by applicant]
US 20190349608A1 · Pollet · 2019 [cited by applicant]
US 20190356718A1 · Haustein et al. · 2019 [cited by applicant]
US 20200014742A1 · Pollet · 2020 [cited by applicant]
GB 2552016A · 2018 [cited by examiner]
JP 2009005114A · 2009 [cited by applicant]
JP 2010068111A · 2010 [cited by applicant]
JP 2012212142A · 2012 [cited by applicant]
WO WO2006110975A1 · 2006 [cited by applicant]
WO WO2014154822A1 · 2014 [cited by applicant]
Jakubisin , “Real-Time Audio Transceiver Utilizing 802.11b Wireless Technology” (Year: 2007). [cited by examiner]
Y. Lu, Z. Qiao, P. Zhang and L. Guo, “Ranking-Based Music Recommendation in Online Music Radios,” 2016 IEEE First International Conference on Data Science in Cyberspace (DSC), Changsha, China, 2016, pp. 614-619 (Year: 2… [cited by examiner]
Dias, R., & Fonseca, M. J. (2013). Improving music recommendation in session-based collaborative filtering by using temporal context doi:http://dx.doi.org/10.1109/ICTAI.2013.120 (Year: 2013). [cited by examiner]
P. C. Bach, N. Vyrva and S. K. Stigberg, “Tangible music player: Music collaboration,” 2015 International Conference on Collaboration Technologies and Systems (CTS), Atlanta, GA, USA, 2015, pp. 399-406 (Year: 2015). [cited by examiner]
M. Levy and M. Sandler, “Music Information Retrieval Using Social Tags and Audio,” in IEEE Transactions on Multimedia, vol. 11, No. 3, pp. 383-395, Apr. 2009 (Year: 2009). [cited by examiner]
Carot et al “Paper Presentation at the ‘Music in the Global Village’ Conference” International School of New Media at the University of Lubeck, 2007. [cited by applicant]
Cutietta “The Internet for Ensemble Performance” 2004. [cited by applicant]
Denon “Wireless, AP or Client Isolation” 2016. [cited by applicant]
Discover New Worlds with MobileConnect, 2014. [cited by applicant]
El-Shimy et al “EmbodiNet: Enriching Distributed Musical Collaboration Through Embodied Interactions” 15 [cited by applicant]
Final Office Action issued in U.S. Appl. No. 15/978,726 on Mar. 28, 2022. [cited by applicant]
Fitzek et al “Robust Header Compression (ROHC) Performance for Multimedia Transmission Over 3G/4G Wireless Networks” Wireless Pers Commun vol. 32, pp. 23-41, 2005. [cited by applicant]
Fong et al “Online eLearning System for Live Band Practicing” 2016 International Symposium on Educational Technology (ISET), pp. 69-72, 2016. [cited by applicant]
Gettelman “Yes Mines Cutting Edge of Sound” Orlando Sentinel, 1994. [cited by applicant]
Giuli et al “Orchestral: A Distributed Platform for Virtual Music Groups and Music Distance Learning Over the Internet in Java™ Technology” Proceedings IEEE International Conference on Multimedia Computing and Systems, … [cited by applicant]
“IP Multicast Network Management” Cisco Systems. [cited by applicant]
Kening “Yes Answers Critics Who Think They're Ready for Scraphead” Chicago Tribute, 1994. [cited by applicant]
Lee “What is the Difference Between UDP and TCP Internet Protocols?” Reliability Handbook, 2019. [cited by applicant]
Multicast with Wireless LAN Controllers (WLCs) and Lightweight Access Points (LAPs) Configuration Example. [cited by applicant]
Pei et al “Why it Takes so Long to Connect to a WiFi Access Point” IEEE Conference on Computer Communications, pp. 1-9, 2017. [cited by applicant]
Sawchuk “From Remote Media Immersion to Distributed Immersive Performance” 2003. [cited by applicant]
Sui et al “Characterizing and Improving WiFi Latency in Large-Scale Operational Networks” pp. 347-360. [cited by applicant]
Viachakis et al “Using a Dedicated Toolkit and the Cloud to Coordinate Shared Music Representations” The 5 [cited by applicant]
“Talk (Yes Album)” Wikipedia Encyclopedia. [cited by applicant]
Facchinetti et al “Real-Time Voice Streaming Over IEEE 802.15.4” the IEEE Symposium on Computers and Communications, pp. 985-990, 2010. [cited by applicant]
Gujar et al “Realtime Audio Streaming Using Visible Light Communication” 2016 International Conference on Inventive Computation Technologies (ICICT), pp. 1-3, 2016. [cited by applicant]
Ichhorn et al “A Realtime Streaming Architecture for In-Car-Multimedia: Design Guidelines and Prototypical Implementation” 2008. [cited by applicant]
International Search Report and Written Opinion issued in International Patent Application No. PCT/US2018/032532 on Oct. 9, 2018. [cited by applicant]
International Search Report and Written Opinion issued in International Patent Application No. PCT/US2018/032535 on Aug. 29, 2018. [cited by applicant]
Invitation to Pay Additional Fees and, Where Applicable, Protest Fee issued in International Patent Application No. PCT/US2018/032532 on Aug. 14, 2018. [cited by applicant]
Kleimola “Latency Issues in Distributed Musical Performance” 2006. [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 15/978,726 on Jul. 13, 2022. [cited by applicant]