IP Library Granted Patent US 10,812,929
Granted Patent B1
US 10,812,929 · App. 16/554,401 · Granted Oct 20, 2020

Inferring pinnae information via beam forming to produce individualized spatial audio

Inventors: Jacob Ryan Donley (Kirkland, WA); Pablo Francisco Faundez Hoffmann (Kenmore, WA)
Assignee: Facebook Technologies, LLC
H04S7/307H04R1/403H04R3/12H04R5/02H04R5/027H04R5/04H04S7/303H04S2400/15H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,812,929
App. No.
16/554,401
Granted
Oct 20, 2020
Kind
B1
Abstract

An audio system presents spatialized audio content to a user that is individually calibrated for the user. The audio system presents sounds to the user, which reflects off the user's ear. An array of acoustic sensors of the audio system generate audio data from the presented sound. The audio system processes the audio data, using beamformers that each point to a respective portion of the ear, to generate beamformed signals. The audio system determines transfer functions that define transformations of the sound caused by reflections off the user's ear, using the beamformed signals. The audio system generates spatialized audio content for the ear based on the transfer functions.

Claims (57)

1. A method comprising:

generating, by acoustic sensors of a sensor array, audio data from one or more sounds received by the acoustic sensors;

generating beamformed signals by processing the audio data using beamformers, each beamformer pointing to a different portion of an ear of a user;

determining transfer functions defining transformations of the sound caused by reflections from the different portions of the ear using the beamformed signals; and

generating spatialized audio content for the ear based on the transfer functions.

2. The method of claim 1 , wherein generating the spatialized audio content based on the transfer functions includes:

determining an at-the-ear equalization filter based on the transfer functions; and

adjusting audio content for the user using the at-the-ear equalization filter.

3. The method of claim 2 , wherein determining the at-the-ear equalization filter includes referencing a database of reference at-the-ear equalization filters.

4. The method of claim 2 , wherein determining the at-the-ear equalization filter includes correlating the transfer functions to a filter calibrated for the user.

5. The method of claim 1 , wherein determining the transfer functions defining transformations of the sound caused by reflections from the portions of the ear using the beamformed signals includes:

generating, by the acoustic sensors of the sensor array, other audio data from one or more other sounds received by the acoustic sensors without reflection from the portions of the ear;

generating calibration signals by processing the other audio data using the beamformers; and

determining the transfer functions using the beamformed signals and the calibration signals.

6. The method of claim 1 , wherein:

at least one acoustic sensor of the sensor array is positioned at an entrance of an ear canal of the ear of the user; and

determining the transfer functions defining transformations of the sound caused by reflections from the portions of the ear using the beamformed signals includes:

generating, by the at least one acoustic sensor of the sensor array, other audio data from one or more other sounds received by the at least one acoustic sensor; and

determining the transfer functions using the beamformed signals and the other audio data.

7. The method of claim 1 , wherein the beamformed signals collectively indicate a measure of sound pressure at a center of the ear of the user.

8. The method of claim 1 , further comprising generating, by at least one transducer, the one or more sounds received by the acoustic sensors.

9. The method of claim 1 , wherein each beamformer points to a different portion of a pinna of the ear.

10. The method of claim 1 , further comprising:

generating a first beamformer of the beamformers pointing to a first portion of the ear;

generating first audio data of the audio content by the acoustic sensors of the sensor array from a first sound of the one or more sounds; and

processing the first audio data using the first beamformer to generate a first beamformed signal of the beamformed signals.

11. An audio system comprising:

a sensor array including acoustic sensors, the acoustic sensors configured to:

generate audio data from one or more sounds received by the acoustic sensors; and

an audio controller configured to:

generate beamformed signals by processing the audio data using beamformers for the acoustic sensors of the sensor array, each beamformer pointing to a different portion of an ear of a user;

determine transfer functions defining transformations of the sound caused by reflections from the different portions of the ear using the beamformed signals; and

generate spatialized audio content for the ear based on the transfer functions.

12. The audio system of claim 11 , wherein the audio controller is further configured to:

determine an at-the-ear equalization filter based on the transfer functions; and

adjust audio content using the at-the-ear equalization filter.

13. The audio system of claim 12 , wherein the audio controller is further configured to reference a database of reference at-the-ear equalization filters.

14. The audio system of claim 12 , wherein the audio controller is further configured to correlate the transfer functions to a filter calibrated for the user.

15. The audio system of claim 11 , wherein the audio controller is further configured to:

generate, by the acoustic sensors of the sensor array, other audio data from one or more other sounds received by the acoustic sensors without reflection from the portions of the ear;

generate calibration signals by processing the other audio data using the beamformers; and

determine the transfer functions using the beamformed signals and the calibration signals.

16. The audio system of claim 11 , wherein:

at least one acoustic sensor of the sensor array is positioned at an entrance of an ear canal of the ear of the user; and

the audio controller is further configured to:

generate, by the at least one acoustic sensor of the sensor array, other audio data from one or more other sounds received by the at least one acoustic sensor; and

determine the transfer functions using the beamformed signals and the other audio data.

17. The audio system of claim 11 , wherein the beamformed signals collectively indicate a measure of sound pressure at a center of the ear of the user.

18. The audio system of claim 11 , wherein each beamformer points to a different portion of a pinna of the ear.

19. A computer readable non-transitory storage medium storing instructions for presenting spatialized audio content, the instructions, when executed by a processor, cause the processor to perform steps comprising:

generating, by acoustic sensors of a sensor array, audio data from one or more sounds received by the acoustic sensors;

generating beamformed signals by processing the audio data using beamformers, each beamformer pointing to a different portion of an ear of a user;

determining transfer functions defining transformations of the sound caused by reflections from the different portions of the ear using the beamformed signals; and

generating spatialized audio content for the ear based on the transfer functions.

20. The computer readable non-transitory storage medium of claim 19 , wherein the instructions further cause the processor to perform steps comprising:

determining an at-the-ear equalization filter based on the transfer functions; and

adjusting audio content for the user using the at-the-ear equalization filter.

Assignments (2)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060315/0224 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 4, 2019
From: DONLEY, JACOB RYAN; FAUNDEZ HOFFMANN, PABLO FRANCISCO
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 050259/0454 →
Cited By (1)
US 12,526,585