IP Library Granted Patent US 12,383,844
Granted Patent B2
US 12,383,844 · App. 17/676,082 · Granted Aug 12, 2025

Hands-free audio control device

Inventors: Eric Justin Gould Bear (Austin, TX); Rachel M. Strickland (San Francisco, CA); Jim McKee (San Francisco, CA)
Assignee: MONKEYmedia, Inc.
A63H30/04A63F13/213A63F13/428A63F13/45A63F13/803A63H33/22G02B27/0172G02B27/0179G05D1/0016G05D1/0038G05D1/0094G05D1/223G05D1/224G05D1/689G06F1/1688G06F1/1694G06F3/012G06F3/0304G06F3/0346G06F3/046G06F3/04815G06F3/167G06T15/10G06T15/20G06T19/003H04N5/445H04N7/15H04N7/157H04N13/204H04N21/41407H04N21/42202H04N21/42224H04N21/4223H04N21/4312H04N21/4316H04N21/47H04N21/8106H04N23/66A63F13/00A63F13/211A63F13/235A63F13/525A63F13/92A63F2300/204A63F2300/8082A63H2200/00G02B2027/014G06F2200/1637G06T2215/16H04N7/142
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,383,844
App. No.
17/676,082
Granted
Aug 12, 2025
Kind
B2
Abstract

Traditional head-mounted display experiences require a hand controller (e.g. thumbstick, touchpad, gamepad, keyboard, etc.) to manipulate audio controls. The invention is a hands-free, body-based navigation technology that puts the participant's body in direct control of movement through sonic space. Participants lean towards a sound source to make it louder and lean away to make it quieter. In some embodiments, the more a participant leans, the faster the volume changes. Because the interactions were designed to respond to natural bearing and balancing instincts, movement coordination is intuitive and easy to learn.

Claims (33)

1. A system for optimizing the audibility of a collection of digital audio streams, comprising:

a head-mounted audio device configured to play a plurality of digital audio streams to a person wearing the head-mounted audio device, the head-mounted audio device being associated with one or more microprocessors and one or more sensors, wherein the head-mounted audio device, in combination with the microprocessors and sensors, is configured to cause at least one of the audio streams having a location in space to increase in volume when the person leans toward said location while causing decrease in the volume of any audio stream with a different location in space that the person is leaning away from.

2. The system of claim 1 , wherein the head-mounted audio device is configured to vary the audio stream's volume in response to a lean angle of the person.

3. The system of claim 2 , wherein the head-mounted audio device is configured to cause the audio stream's increase in volume to be greater as the person leans at a greater angle toward the location of the audio stream.

4. The system of claim 3 , wherein the audio stream's increased volume causes said audio stream to appear closer to the person.

5. The system of claim 2 , wherein the head-mounted audio device is configured cause any audio stream's decrease in volume to be greater as the person leans at a greater angle away from its different location.

6. The system of claim 5 , wherein any audio stream's decreased volume causes said audio stream to appear further away from the person.

7. The system of claim 2 , wherein the head-mounted audio device is configured to cause the audio stream to maintain a level of increased volume when the person leans toward the location of the audio stream past a specified threshold lean angle.

8. The system of claim 2 , wherein the head-mounted audio device is configured to cause any audio stream to maintain a level decreased volume when the person leans away from its different location past a specified threshold lean angle.

9. The system of claim 2 , wherein the one or more digital audio streams are played in a virtual environment.

10. The system of claim 2 , wherein the one or more digital audio streams are spatialized over a soundscape of the physical world.

11. A method for optimizing the audibility of a collection of digital audio streams, comprising:

playing, by a computer processor associated with one or more sensors, one or more digital audio streams to a person wearing a head-mounted audio device; and

causing, by the computer processor, at least one of the audio streams having a location in space to increase in volume when the person leans toward said location while causing a decrease in the volume of any audio stream with a different location in space that the person is leaning away from.

12. The method of claim 11 , further comprising varying, by the computer processor, the audio stream's volume in response to a lean angle of the person.

13. The method of claim 12 , further comprising causing the audio stream's increase in volume to be greater as the person leans at a greater angle toward the location of the audio stream.

14. The method of claim 13 , wherein the audio stream's increased volume causes said audio stream to appear closer to the person.

15. The method of claim 12 , further comprising causing any audio stream's decrease in volume to be greater as the person leans at a greater angle away from its different location.

16. The method of claim 15 , wherein any audio stream's decreased volume causes said audio stream to appear further away from the person.

17. The method of claim 12 , further comprising maintaining a level of increased volume of the audio stream when the person leans toward the location of the audio stream past a specified threshold lean angle.

18. The method of claim 12 , further comprising maintaining a level of decreased volume of any audio stream when the person leans away from its different location past a specified threshold lean angle.

19. The method of claim 12 , further comprising playing the one or more digital audio streams in a virtual environment.

20. The method of claim 12 , further comprising spatializing the one or more digital audio streams over a soundscape of the physical world.

21. One or more computer readable media (CRM) comprising instructions that, when executed by a computer associated with a head-mounted audio device and one or more sensors, are capable of playing a collection of digital audio streams to a person and causing at least one of the audio streams to having a location in space to increase in volume when the person leans toward said location while causing a decrease in the volume of any audio stream with a different location in space that the person is leaning away from.

22. The CRM of claim 21 , wherein the instructions are capable of varying the audio stream's volume in response to a lean angle of the person.

23. The CRM of claim 21 , wherein the instructions are capable of causing the audio stream's increase in volume to be greater as the person leans at a greater angle toward the location of the audio stream.

24. The CRM of claim 23 , wherein the audio stream's increased volume causes said audio stream to appear closer to the person.

25. The CRM of claim 21 , wherein the instructions are capable of causing any audio stream's decrease in volume to be greater as the person leans at a greater angle away from its different location.

26. The CRM of claim 25 , wherein any audio stream's decreased volume causes said audio stream to appear further away from the person.

27. The CRM of claim 21 , wherein the instructions are capable of causing the audio stream to maintain a level of increased volume when the person leans toward the location of the audio stream past a specified threshold lean angle.

28. The CRM of claim 21 , wherein the instructions are capable of causing any audio stream to maintain a level of decreased volume when the person leans away from its different location past a specified threshold lean angle.

29. The CRM of claim 21 , wherein the instructions are capable of causing the one or more digital audio streams to be played in a virtual environment.

30. The CRM of claim 21 , wherein the instructions are capable of causing the one or more digital audio streams to be spatialized over a soundscape of the physical world.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2023
From: BEAR, ERIC JUSTIN GOULD; STRICKLAND, RACHEL M.; MCKEE, JIM
To: MONKEYMEDIA, INC.
Reel/Frame 064398/0656 →
Continuity (8)
Continuation 16793056 · Feb 18, 2020
Continuation In Part 15905182 · Feb 26, 2018
Continuation 15694210 · Sep 1, 2017
Continuation 15426697 · Feb 7, 2017
Continuation 15186793 · Jun 20, 2016
Continuation 13540695 · Jul 3, 2012
Provisional Application 61666216 · Jun 29, 2012
Related Publication 20220168660A1 · Jun 2, 2022
References Cited (102)
US 5602566A · Motosyuku et al. · 1997 [cited by applicant]
US 5692212A · Roach · 1997 [cited by applicant]
US 5904724A · Margolin · 1999 [cited by applicant]
US 6201544B1 · Lands · 2001 [cited by applicant]
US 6285392B1 · Satoda et al. · 2001 [cited by applicant]
US 6393158B1 · Gould et al. · 2002 [cited by applicant]
US 6621980B1 · Gould et al. · 2003 [cited by applicant]
US 7004838B2 · Hayashida et al. · 2006 [cited by applicant]
US 7467218B2 · Gould et al. · 2008 [cited by applicant]
US 7625287B2 · Champagne et al. · 2009 [cited by applicant]
US 7631277B1 · Nie et al. · 2009 [cited by applicant]
US 7688306B2 · Wehrenberg et al. · 2010 [cited by applicant]
US 7865834B1 · van Os et al. · 2011 [cited by applicant]
US 7890648B2 · Gould et al. · 2011 [cited by applicant]
US 8122143B2 · Gould et al. · 2012 [cited by applicant]
US 8269175B2 · Alameh et al. · 2012 [cited by applicant]
US 8571192B2 · Etter · 2013 [cited by examiner]
US 8634969B2 · Buelthoff et al. · 2014 [cited by applicant]
US 8797260B2 · Mao et al. · 2014 [cited by applicant]
US 8872854B1 · Levitt · 2014 [cited by applicant]
US 8907943B2 · Piemonte · 2014 [cited by applicant]
US 8928696B1 · Yang · 2015 [cited by applicant]
US 9001154B2 · Meier et al. · 2015 [cited by applicant]
US 9185379B2 · Gould et al. · 2015 [cited by applicant]
US 9441985B2 · Riley · 2016 [cited by applicant]
US 9563202B1 · Bear et al. · 2017 [cited by applicant]
US 9563270B2 · Tichenor et al. · 2017 [cited by applicant]
US 9579586B2 · Bear et al. · 2017 [cited by applicant]
US 9612627B2 · Bear et al. · 2017 [cited by applicant]
US 9656168B1 · Bear et al. · 2017 [cited by applicant]
US 9658617B1 · Bear et al. · 2017 [cited by applicant]
US 9782684B2 · Bear et al. · 2017 [cited by applicant]
US 9791897B2 · Bear et al. · 2017 [cited by applicant]
US 10298911B2 · Shuster · 2019 [cited by examiner]
US 10596478B2 · Bear et al. · 2020 [cited by applicant]
US 20030067535A1 · Creed · 2003 [cited by examiner]
US 20040040434A1 · Kondo · 2004 [cited by examiner]
US 20040169676A1 · Williams et al. · 2004 [cited by applicant]
US 20070024644A1 · Bailey · 2007 [cited by applicant]
US 20070035563A1 · Biocca et al. · 2007 [cited by applicant]
US 20070149282A1 · Lu et al. · 2007 [cited by applicant]
US 20070265087A1 · Akahori et al. · 2007 [cited by applicant]
US 20080063400A1 · Hudson et al. · 2008 [cited by applicant]
US 20080070684A1 · Haigh-Hutchinson · 2008 [cited by applicant]
US 20090034766A1 · Hamanaka · 2009 [cited by examiner]
US 20100328423A1 · Etter · 2010 [cited by examiner]
US 20110002487A1 · Panther · 2011 [cited by examiner]
US 20110018903A1 · Lapstun et al. · 2011 [cited by applicant]
US 20110059777A1 · Rao · 2011 [cited by applicant]
US 20110113383A1 · Dyack et al. · 2011 [cited by applicant]
US 20110161890A1 · Anderson et al. · 2011 [cited by applicant]
US 20120002024A1 · Choi · 2012 [cited by examiner]
US 20120040759A1 · Ito et al. · 2012 [cited by applicant]
US 20120218301A1 · Miller · 2012 [cited by applicant]
US 20120272175A1 · Lin · 2012 [cited by applicant]
US 20130141421A1 · Mount · 2013 [cited by examiner]
US 20140002580A1 · Bear et al. · 2014 [cited by applicant]
US 20140002581A1 · Bear et al. · 2014 [cited by applicant]
US 20140002582A1 · Bear et al. · 2014 [cited by applicant]
US 20140063054A1 · Osterhout et al. · 2014 [cited by applicant]
US 20140063055A1 · Osterhout et al. · 2014 [cited by applicant]
US 20140118495A1 · Coombe · 2014 [cited by applicant]
US 20140297098A1 · Yoshimoto et al. · 2014 [cited by applicant]
US 20150302756A1 · Guehring et al. · 2015 [cited by applicant]
US 20160194079A1 · Montreuil · 2016 [cited by applicant]
GB 2378878A · 2003 [cited by applicant]
JP 2000153057A · 2000 [cited by examiner]
JP 2009514312A · 2009 [cited by examiner]
JP 2012514392A · 2012 [cited by examiner]
JP 2020014532A · 2020 [cited by examiner]
KR 20120051209A · 2012 [cited by examiner]
WO 9815920 · 1998 [cited by applicant]
WO 0186920A2 · 2001 [cited by applicant]
WO 03001340A2 · 2003 [cited by applicant]
WO 2008094458A1 · 2008 [cited by applicant]
WO WO2021262303A1 · 2021 [cited by examiner]
Non-Final Office Action, U.S. Appl. No. 15/186,776, mailed Aug. 18, 2016, 23 pages. [cited by applicant]
Pre-Interview First Office Action, U.S. Appl. No. 15/186,783, mailed Sep. 30, 2016, 17 pages. [cited by applicant]
Pre-Interview First Office Action, U.S. Appl. No. 15/186,793, mailed Sep. 23, 2016, 5 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/186,783, mailed Dec. 1, 2016, 25 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/186,793, mailed Dec. 27, 2016, 39 pages. [cited by applicant]
Curriculum Vitae and Declaration of Nathan Shedroff under 37 CFR 1.132 regarding the 12A, B & C patent. Applications as submitted with an Appeal Brief during prosecution of Co-Pending U.S. Appl. No. 13/540,695 entitled,… [cited by applicant]
“Super Monkey Ball 2: Sakura Edition,” SEGA Corporation <http://itunes.apple.com/us/app/super-monkey-ball-2-sakura/id363644188> and tutorial screenshots (captured Jun. 7, 2012). [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 13/541,388 mailed on Jul. 22, 2014. [cited by applicant]
Applicant-Initiated Interview Summary for U.S. Appl. No. 13/541,388 mailed on Nov. 21, 2014. [cited by applicant]
Scagliarini et al. “Exciting understanding in Pompeii through on-site parallel interaction with dual time virtual models” VAST '01 Proceedings of the 2001 Conference on Virtual Reality, Archeology, and Cultural Heritage… [cited by applicant]
First Action Interview Pilot Program Pre-Interview Communication for U.S. Appl. No. 13/541,337 mailed on Jul. 22, 2014. [cited by applicant]
Applicant-Initiated Interview Summary for U.S. Appl. No. 13/541,337 mailed on Nov. 21, 2014. [cited by applicant]
First Action Interview Pilot Program Pre-Interview Communication for U.S. Appl. No. 13/540,695 mailed on Dec. 17, 2012. [cited by applicant]
Applicant-Initiated Interview Summary for U.S. Appl. No. 13/540,695 mailed on Mar. 19, 2013. [cited by applicant]
First Action Interview Office Action Summary for U.S. Appl. No. 13/540,695 mailed on Mar. 19, 2013. [cited by applicant]
Marsden et al., Navigation Control for Mobile Virtual Environments, MobileHCI '05 Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices and Services, Sep. 2005, pp. 279-282. [cited by applicant]
Hurst et al., Mobile 3D Graphics and Virtual Reality Interaction, ACE '11 Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology, Nov. 2011. [cited by applicant]
Final Office Action Summary for U.S. Appl. No. 13/540,695 mailed on Jun. 14, 2013. [cited by applicant]
Final Office Action Summary for U.S. Appl. No. 13/540,695 mailed on Apr. 24, 2014. [cited by applicant]
Examiner's Answer to Appeal Brief for U.S. Appl. No. 13/540,695 mailed on May 18, 2015. [cited by applicant]
Walking improves your cognitive map in environments that are large-scale and large in extent, Roy A. Ruddle, Ekaterina Volkova and Heinrich H. Bi.ilthoff, ACM Transactions on Computer-Human Interaction, v. 18 n.2, p. 1-… [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/186,776, mailed Feb. 13, 2017, 32 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/694,210, mailed Jan. 24, 2018, 6 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/425,666, mailed Apr. 14, 2017, 9 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/426,697, mailed Aug. 23, 2017, 10 pages. [cited by applicant]
Jay et al. “Amplifying Head Movements with Head-Mounted Displays”, The MIT Press, vol. 12/Issue Mar. 2003 (Year 2003). [cited by applicant]