IP Library › Granted Patent US 10,412,527
Granted Patent B1
US 10,412,527 · App. 15/865,522 · Granted Sep 10, 2019

Head-related transfer function determination using base stations

Inventors: Antonio John Miller (Sammamish, WA); Ravish Mehra (Redmond, WA)
Assignee: Facebook Technologies, LLC
H04S7/301H04N13/204H04R5/02H04S3/008H04S7/303H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,412,527
App. No.
15/865,522
Filed
Jan 9, 2018
Granted
Sep 10, 2019
Kind
B1
Art Unit
2654
USPC
381/303
Abstract

A system including base stations determines head-related transfer functions (HRTFs) for a user. Each base station is located at a distinct location within a local area and includes a speaker configured to emit a test sound in accordance with calibration instructions. A depth camera assembly determines depth information describing a position of a head-mounted display (HMD) in the local area relative to the locations of the base stations. A microphone is placed in an ear canal of a user wearing the HMD, and generates a respective audio sample from the test sound emitted by the speaker of each base station. A controller determines the relative position of the HMD using the depth information, generates the calibration instructions based on the relative position of the HMD, and determines the HRTFs based on the audio samples.

Claims (76)

1. A system comprising:

a plurality of base stations, each base station located at a respective distinct static location within a local area and including a speaker configured to emit a test sound in accordance with calibration instructions;

a depth camera assembly configured to determine depth information describing a position of a head-mounted display (HMD) in the local area relative to the locations of the plurality of base stations;

a microphone configured to:

be placed in an ear canal of a user wearing the HMD, and

generate a respective audio sample from the test sound emitted by the speaker of each base station; and

a controller configured to:

determine the relative position of the HMD using the depth information;

generate the calibration instructions based on the relative position of the HMD; and

determine a head related transfer function (HRTF) based on the audio samples.

2. The system of claim 1 , wherein the microphone is coupled to the HMD.

3. The system of claim 1 , wherein the depth camera assembly comprises:

a light projector configured to provide structured light in a direction of the HMD;

an imaging device configured to capture images of the HMD and the plurality of base stations; and

a depth camera controller configured to determine the depth information using the captured images.

4. The system of claim 1 , wherein the determining of the HRTF comprises steps to:

determine a first transfer function based on audio samples generated by the microphone coupled to the HMD;

determine a second transfer function based on audio samples generated by the microphone coupled to the HMD worn by the user; and

compute the HRTF as a difference between the first transfer function and the second transfer function.

5. The system of claim 1 , wherein the test sounds comprise one or more logarithmic chirps.

6. The system of claim 1 , wherein the controller is further configured to:

populate a set of configurations, each configuration comprising the location of at least one base station;

for each configuration, instruct the speaker of the at least one base station to emit the test sound to obtain the audio samples from the microphone for the configuration; and

determine HRTFs based on the audio samples for the set of configurations.

7. The system of claim 6 , wherein the controller is further configured to:

determine an audio characterization configuration based on the HRTFs, the audio characterization configuration being a function to synthesize a binaural sound that seems to come from a particular point in a virtual space; and

synthesizing the binaural sound based on the audio characterization configuration.

8. A head-mounted display (HMD) comprising:

a microphone configured to:

be placed in an ear canal of a user wearing the HMD, and

generate respective audio samples from test sounds received from a plurality of base stations, each base station located at a respective distinct static location within a local area and including a speaker configured to emit a test sound in accordance with calibration instructions; and

a controller configured to:

determine depth information describing a position of the HMD in the local area relative to the locations of the plurality of base stations using images captured by a depth camera assembly,

determine the position of the HMD using the depth information,

generate the calibration instructions based on the relative position of the HMD, and

determine a head-related transfer function (HRTF) based on the audio samples.

9. The HMD of claim 8 , wherein the depth camera assembly comprises:

a light projector configured to provide structured light in a direction of the HMD; and

an imaging device configured to capture the images of the HMD and the plurality of base stations.

10. The HMD of claim 8 , wherein the determining of the HRTF comprises steps to:

determine a first transfer function based on audio samples generated by the microphone;

determine a second transfer function based on audio samples generated by the microphone worn by the user; and

compute the HRTF as a difference between the first transfer function and the second transfer function.

11. The HMD of claim 8 , wherein the test sounds comprise one or more logarithmic chirps.

12. The HMD of claim 8 , wherein the controller is further configured to:

populate a set of configurations, each configuration comprising the location of at least one base station;

for each configuration, instruct the speaker of the at least one base station to emit the test sound to obtain the audio samples from the microphone for the configuration; and

determine HRTFs based on the audio samples for the set of configurations.

13. The HMD of claim 12 , wherein the controller is further configured to:

determine an audio characterization configuration based on the HRTFs, the audio characterization configuration being a function to synthesize a binaural sound that seems to come from a particular point in a virtual space; and

synthesizing the binaural sound based on the audio characterization configuration.

14. A method comprising:

determining depth information describing a position of a head-mounted display (HMD) in a local area relative to a plurality of base stations, each base station located at a respective distinct static location within the local area;

determining the relative position of the HMD using the depth information;

generating calibration instructions based on the relative position of the HMD;

emitting test sounds in accordance with the calibration instructions from speakers of the plurality of base stations;

generating, by a microphone configured to be placed in an ear canal of a user wearing the HMD, respective audio samples from the test sounds emitted by the speakers;

determining a head-related transfer function (HRTF) based on the audio samples.

15. The method of claim 14 , wherein the microphone is coupled to the HMD.

16. The method of claim 14 , wherein the determining of the depth information comprises:

illuminating, by a light projector, the HMD with structured light, wherein the light projector is coupled to a first base station of the plurality of base stations;

capturing, by an imaging device, images of the HMD, wherein the imaging device is coupled to the first base station; and

determining the depth information based in part on the captured images.

17. The method of claim 14 , wherein the determining of the HRTF comprises:

determining a first transfer function based on audio samples generated by the microphone coupled to the HMD;

determining a second transfer function based on audio samples generated by the microphone coupled to the HMD worn by the user; and

computing the HRTF as a difference between the first transfer function and the second transfer function.

18. The method of claim 14 , wherein the test sounds comprise one or more logarithmic chirps.

19. The method of claim 14 , further comprising:

populating a set of configurations, each configuration comprising the location of at least one base station;

for each configuration, instructing the speaker of the at least one base station to emit the test sound to obtain the audio samples from the microphone for the configuration; and

determining HRTFs based on the audio samples for the set of configurations.

20. The method of claim 19 , further comprising:

determining an audio characterization configuration based on the HRTFs, the audio characterization configuration being a function to synthesize a binaural sound that seems to come from a particular point in a virtual space; and

synthesizing the binaural sound based on the audio characterization configuration.

21. The system of claim 1 , wherein the plurality of base stations includes a first base station and a second base station, and a speaker of the first base station and a speaker of the second base station are configured to emit at least one test sound simultaneously.

Assignments (3)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060315/0224 →
CHANGE OF NAME Recorded Sep 12, 2018
From: OCULUS VR, LLC
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047178/0616 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 22, 2018
From: MILLER, ANTONIO JOHN; MEHRA, RAVISH
To: OCULUS VR, LLC
Reel/Frame 044692/0894 →
Cited By (2)
US 12,267,664 US 12,640,024