IP Library Granted Patent US 11,627,428
Granted Patent B2
US 11,627,428 · App. 17/190,326 · Granted Apr 11, 2023

Immersive audio platform

Inventors: Jean-Marc Jot (Aptos, CA); Michael Minnick (Fort Lauderdale, FL); Dmitry Pastouchenko (Folsom, CA); Michael Aaron Simon (Fort Lauderdale, CA); John Emmitt Scott, III (Plantation, FL); Richard St. Clair Bailey (Plantation, FL); Shivakumar Balasubramanyam (San Diego, CA); Harsharaj Agadi (Plantation, FL)
H04S7/304G02B27/017H04S2400/01H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,627,428
App. No.
17/190,326
Granted
Apr 11, 2023
Kind
B2
Abstract

Disclosed herein are systems and methods for presenting audio content in mixed reality environments. A method may include receiving a first input from an application program; in response to receiving the first input, receiving, via a first service, an encoded audio stream; generating, via the first service, a decoded audio stream based on the encoded audio stream; receiving, via a second service, the decoded audio stream; receiving a second input from one or more sensors of a wearable head device; receiving, via the second service, a third input from the application program, wherein the third input corresponds to a position of one or more virtual speakers; generating, via the second service, a spatialized audio stream based on the decoded audio stream, the second input, and the third input; presenting, via one or more speakers of the wearable head device, the spatialized audio stream.

Claims (68)

1. A system comprising:

a wearable head device comprising one or more sensors and one or more speakers, the wearable head device in a first position; and

one or more processors configured to execute a method comprising:

receiving a first input from an application program;

in response to receiving the first input, receiving, via a first service, an encoded audio stream;

generating, via the first service, a decoded audio stream based on the encoded audio stream;

receiving, via a second service, the decoded audio stream;

receiving a second input from one or more sensors of the wearable head device;

receiving, via the second service, a third input from the application program, wherein the third input corresponds to a position of one or more virtual speakers;

generating, via the second service, a spatialized audio stream based on the decoded audio stream, the second input, and the third input;

in accordance with a determination that the spatialized audio stream is associated with a per-screen spatialized audio:

presenting, via the one or more speakers of the wearable head device, the spatialized audio stream, wherein:

the spatialized audio stream is associated with one of a static mode and a dynamic mode,

the wearable head device moves from the first position to a second position, different from the first position; and

in accordance with a determination that the spatialized audio stream is associated with the static mode, presenting the first spatialized audio based on a first distance from the first position to the position of the one or more virtual speakers; and

in accordance with a determination that the spatialized audio stream is associated with the dynamic mode, presenting a second spatialized audio based on a second distance from the second position to the position of the one or more virtual speakers.

2. The system of claim 1 , wherein the second input corresponds to a position of the wearable head device.

3. The system of claim 2 , wherein the position of the one or more virtual speakers is based on the position of the wearable head device.

4. The system of claim 1 , wherein the second input corresponds to a position of a physical object in a real environment.

5. The system of claim 4 , wherein the position of the one or more virtual speakers is based on the position of the physical object.

6. The system of claim 1 , wherein the encoded audio stream is received from a remote server.

7. The system of claim 1 , wherein the first service is a different service than the second service.

8. The system of claim 1 , wherein the spatialized audio stream comprises six audio channels, wherein one of the six audio channels comprises a subwoofer channel.

9. The system of claim 1 , wherein the spatialized audio stream comprises eight audio channels, wherein one of the eight audio channels comprises a subwoofer channel.

10. The system of claim 1 , wherein presenting the spatialized audio stream is based on a position and distance of the wearable head device relative to a virtual screen configured to present a virtual content displayed on a display of the wearable head device.

11. The system of claim 1 , wherein the wearable head device moves from the first position to a second position, different from the first position, and the method further comprises:

in accordance with a determination that the spatialized audio stream is associated with a per-room spatialized audio:

presenting, via the one or more speakers of the wearable head device, the spatialized audio stream;

in accordance with a determination that the second position is in a first room, presenting a third spatialized audio associated with the first room; and

in accordance with a determination that the second position is in a second room, presenting a fourth spatialized audio associated with the second room,

wherein:

the third spatialized audio is different than the fourth spatialized audio, and

the first room is different than the second room.

12. The system of claim 1 , wherein the presentation of the spatialized audio stream is based on user settings.

13. The system of claim 1 , wherein the presentation of the spatialized audio stream is based on settings of the application program.

14. The system of claim 1 , wherein the first spatialized audio stream is associated with a centralized setting, and the method further comprises:

receiving a fourth input from a second application program; and

presenting, via the one or more speakers of the wearable head device, a second spatialized audio stream associated with the centralized setting.

15. A method comprising:

receiving a first input from an application program;

in response to receiving the first input, receiving, via a first service, an encoded audio stream;

generating, via the first service, a decoded audio stream based on the encoded audio stream;

receiving, via a second service, the decoded audio stream;

receiving a second input from one or more sensors of a wearable head device;

receiving, via the second service, a third input from the application program, wherein the third input corresponds to a position of one or more virtual speakers;

generating, via the second service, a spatialized audio stream based on the decoded audio stream, the second input, and the third input;

in accordance with a determination that the spatialized audio stream is associated with a per-screen spatialized audio:

presenting, via one or more speakers of the wearable head device, the spatialized audio stream, wherein:

the spatialized audio stream is associated with one of a static mode and a dynamic mode,

the wearable head device moves from a first position to a second position, different from the first position; and

in accordance with a determination that the spatialized audio stream is associated with the static mode, presenting the first spatialized audio based on a first distance from the first position to the position of the one or more virtual speakers; and

in accordance with a determination that the spatialized audio stream is associated with the dynamic mode, presenting a second spatialized audio based on a second distance from the second position to the position of the one or more virtual speakers.

16. The method of claim 15 , wherein the second input corresponds to a position of the wearable head device.

17. The method of claim 15 , wherein the second input corresponds to a position of a physical object in a real environment.

18. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to execute a method comprising:

receiving a first input from an application program;

in response to receiving the first input, receiving, via a first service, an encoded audio stream;

generating, via the first service, a decoded audio stream based on the encoded audio stream;

receiving, via a second service, the decoded audio stream;

receiving a second input from one or more sensors of a wearable head device;

receiving, via the second service, a third input from the application program, wherein the third input corresponds to a position of one or more virtual speakers;

generating, via the second service, a spatialized audio stream based on the decoded audio stream, the second input, and the third input; and

in accordance with a determination that the spatialized audio stream is associated with a per-screen spatialized audio:

presenting, via one or more speakers of the wearable head device, the spatialized audio stream, wherein:

the spatialized audio stream is associated with one of a static mode and a dynamic mode,

the wearable head device moves from a first position to a second position, different from the first position; and

in accordance with a determination that the spatialized audio stream is associated with the static mode, presenting the first spatialized audio based on a first distance from the first position to the position of the one or more virtual speakers; and

in accordance with a determination that the spatialized audio stream is associated with the dynamic mode, presenting a second spatialized audio based on a second distance from the second position to the position of the one or more virtual speakers.

Assignments (4)
SECURITY INTEREST Recorded Oct 15, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073109/0238 →
EMPLOYMENT AGREEMENT Recorded Sep 20, 2023
From: BALASUBRAMANYAM, SHIVAKUMAR
To: MAGIC LEAP, INC.
Reel/Frame 064963/0053 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2023
From: JOT, JEAN-MARC; MINNICK, MICHAEL; PASTOUCHENKO, DMITRY; SIMON, MICHAEL AARON; SCOTT III, JOHN EMMITT; BAILEY, RICHARD ST. CLAIR; AGADI, HARSHARAJ
To: MAGIC LEAP, INC.
Reel/Frame 064973/0945 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
Continuity (4)
Provisional Application 63036270 · Jun 8, 2020
Provisional Application 62984198 · Mar 2, 2020
Related Publication 20210185471A1 · Jun 17, 2021
Related Publication 20220386065A2 · Dec 1, 2022