IP Library Granted Patent US 11,212,636
Granted Patent B2
US 11,212,636 · App. 16/970,324 · Granted Dec 28, 2021

Dual listener positions for mixed reality

Inventor: Anastasia Andreyevna Tajik (Fort Lauderdale, FL)
Assignee: Magic Leap, Inc.
H04S7/303G02B27/0101G02B27/0172G06T19/006H04R1/403H04R3/12G02B2027/0138H04R2430/20H04R2499/15H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,212,636
App. No.
16/970,324
Granted
Dec 28, 2021
Kind
B2
Abstract

A method of presenting audio comprises: identifying a first ear listener position and a second ear listener position in a mixed reality environment; identifying a first virtual sound source in the mixed reality environment; identifying a first object in the mixed reality environment; determining a first audio signal in the mixed reality environment, wherein the first audio signal originates at the first virtual sound source and intersects the first ear listener position; determining a second audio signal in the mixed reality environment, wherein the second audio signal originates at the first virtual sound source, intersects the first object, and intersects the second ear listener position; determining a third audio signal based on the second audio signal and the first object; presenting, to a first ear of a user, the first audio signal; and presenting, to a second ear of the user, the third audio signal.

Claims (47)

1. A method of presenting audio signals in a mixed reality environment, the method comprising:

identifying a first ear listener position in the mixed reality environment;

identifying a second ear listener position in the mixed reality environment;

identifying a first virtual sound source in the mixed reality environment;

identifying a first object in the mixed reality environment;

determining a first audio signal in the mixed reality environment, wherein the first audio signal originates at the first virtual sound source and intersects the first ear listener position;

determining a second audio signal in the mixed reality environment, wherein the second audio signal originates at the first virtual sound source, intersects the first object, and intersects the second ear listener position;

determining a third audio signal based on the second audio signal and the first object;

presenting, via a first speaker to a first ear of a user, the first audio signal; and

presenting, via a second speaker to a second ear of the user, the third audio signal.

2. The method of claim 1 , wherein determining the third audio signal from the second audio signal comprises applying a low-pass filter to the second audio signal, the low-pass filter having a parameter based on the first object.

3. The method of claim 1 , wherein determining the third audio signal from the second audio signal comprises applying an attenuation to the second audio signal, a strength of the attenuation based on the first object.

4. The method of claim 1 , wherein identifying the first object comprises identifying a real object.

5. The method of claim 4 , wherein identifying the real object comprises using a sensor to determine a position of the real object relative to the user in the mixed reality environment.

6. The method of claim 5 , wherein the sensor comprises a depth camera.

7. The method of claim 4 , further comprising generating helper data corresponding to the real object.

8. The method of claim 4 , further comprising generating a virtual object corresponding to the real object.

9. The method of claim 1 , further comprising identifying a second virtual object, wherein the first audio signal intersects the second virtual object and a fourth audio signal is determined based on the second virtual object.

10. A system comprising:

a wearable head device comprising:

a display for displaying a mixed reality environment to a user, the display comprising a transmissive eyepiece through which a real environment is visible;

a first speaker configured to present audio signals to a first ear of the user; and

a second speaker configured to present audio signals to a second ear of the user; and

one or more processors configured to perform:

identifying a first ear listener position in the mixed reality environment;

identifying a second ear listener position in the mixed reality environment;

identifying a first virtual sound source in the mixed reality environment;

identifying a first object in the mixed reality environment;

determining a first audio signal in the mixed reality environment, wherein the first audio signal originates at the first virtual sound source and intersects the first ear listener position;

determining a second audio signal in the mixed reality environment, wherein the second audio signal originates at the first virtual sound source, intersects the first object, and intersects the second ear listener position;

determining a third audio signal based on the second audio signal and the first object;

presenting, via a first speaker to the first ear, the first audio signal; and

presenting, via a second speaker to the second ear, the third audio signal.

11. The system of claim 10 , wherein determining the third audio signal from the second audio signal comprises applying a low-pass filter to the second audio signal, the low-pass filter having a parameter based on the first object.

12. The system of claim 10 , wherein determining the third audio signal from the second audio signal comprises applying an attenuation to the second audio signal, a strength of the attenuation based on the first object.

13. The system of claim 10 , wherein identifying the first object comprises identifying a real object.

14. The system of claim 13 , wherein the wearable head device further comprises a sensor, and wherein identifying the real object comprises using the sensor to determine a position of the real object relative to the user in the mixed reality environment.

15. The system of claim 14 , wherein the sensor comprises a depth camera.

16. The system of claim 13 , the one or more processors further configured to perform generating helper data corresponding to the real object.

17. The system of claim 13 , the one or more processors further configured to perform generating a virtual object corresponding to the real object.

18. The system of claim 10 , the one or more processors further configured to perform identifying a second virtual object, wherein the first audio signal intersects the second virtual object and a fourth audio signal is determined based on the second virtual object.

19. The method of claim 1 , further comprising:

determining a fourth audio signal in the mixed reality environment, wherein the fourth audio originates at the first virtual sound source and intersects the second ear listener position without intersecting the first object; and

presenting, via the second speaker to the second ear of the user, the fourth audio signal.

20. The system of claim 10 , the one or more processors further configured to perform:

determining a fourth audio signal in the mixed reality environment, wherein the fourth audio originates at the first virtual sound source and intersects the second ear listener position without intersecting the first object; and

presenting, via the second speaker to the second ear of the user, the fourth audio signal.

Assignments (2)
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2021
From: TAJIK, ANASTASIA ANDREYEVNA
To: MAGIC LEAP, INC.
Reel/Frame 057005/0204 →