IP Library Granted Patent US 11,751,003
Granted Patent B1
US 11,751,003 · App. 17/498,554 · Granted Sep 5, 2023

Personalization of head-related transfer function

Inventors: William Owen Brimijoin, II (Kirkland, WA); Tomasz Rudzki (York, GB); Sebastià Vicenç Amegnual Gari (Seattle, WA); Michaela Warnecke (Somerville, MA); Andrew Francl (Redmond, WA)
Assignee: Meta Platforms Technologies, LLC
H04S7/40H04S7/303H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,751,003
App. No.
17/498,554
Granted
Sep 5, 2023
Kind
B1
Abstract

Embodiments relate to personalization of a head-related transfer function (HRTF) for a given user. A sound source is spatialized for an initial position using an initial version of a HRTF to obtain an initial spatialized sound source. Upon presentation of the initial spatialized sound source, at least one property of the HRTF is adjusted in an iterative manner based on at least one perceptive response from the user to generate a version of the HRTF customized for the user. Each perceptive response from the user indicates a respective offset between a perceived position and a target position of the sound source. The customized version of the HRTF is applied to one or more audio channels to form spatialized audio content for the perceived position. The spatialized audio content is presented to the user, wherein the offset between the perceived position and the target position is reduced.

Claims (62)

1. A method comprising:

spatializing a sound source for an initial position in a local area using an initial version of a head-related transfer function (HRTF) to obtain an initial spatialized sound source;

upon presenting the initial spatialized sound source to a user, adjusting, in an iterative manner based on at least one perceptive response from the user, at least one property of the HRTF to generate a version of the HRTF customized for the user, each perceptive response from the user indicating a respective offset between a perceived position and a target position of the sound source upon presentation of at least one spatialized version of the sound source, wherein adjusting the at least one property of the HRTF comprises:

pointing, by the user via an interface device, to at least one location in the local area as at least one perceived location of the sound source, the at least one location being outside of a field of view of the user, and

extrapolating, based on the at least one pointed location and using a machine learning (ML) model, one or more parameters associated with the HRTF to generate the customized version of the HRTF;

applying the customized version of the HRTF to one or more audio channels to form spatialized audio content for the perceived position; and

presenting the spatialized audio content to the user, wherein the offset between the perceived position and the target position is reduced.

2. The method of claim 1 , further comprising:

spatializing the sound source for the initial position using an audio renderer, the audio renderer approximating head-related transfer functions (HRTFs) for the user, and the approximation is based on values of a plurality of parameters used by the audio renderer;

presenting the initial spatialized sound source using the values of the plurality of parameters;

adjusting, in the iterative manner based on the at least one perceptive response from the user, the values of the plurality of parameters to reduce the offset;

spatializing the sound source for the perceived position using the audio renderer configured with the adjusted values of the plurality of parameters; and

presenting the sound source spatialized with the adjusted values, wherein the offset between the perceived position and the target position is reduced.

3. The method of claim 1 , wherein the initial version of the HRTF is a generic HRTF or a non-individualized HRTF.

4. The method of claim 1 , further comprising:

predicting at least one parameter of the HRTF that forms the initial version of the HRTF individualized for the user.

5. The method of claim 1 , further comprising:

selecting the initial version of the HRTF from a set of HRTFs based on one or more features of the user.

6. The method of claim 1 , wherein each perceptive response from the user further indicates a change in an apparent coloration of a sound from the sound source, and the method further comprising:

presenting the spatialized audio content to the user, wherein the apparent coloration in the presented spatialized audio content is reduced below a threshold level.

7. The method of claim 1 , wherein adjusting the at least one property of the HRTF further comprises:

warping at least one of an interaural time difference (ITD) of the HRTF and a spectrum of the HRTF to generate the customized version of the HRTF, based on the at least one perceptive response from the user.

8. The method of claim 1 , wherein adjusting the at least one property of the HRTF further comprises:

adjusting at least one of an amplitude level, a frequency, and a quality factor of at least one biquad filter associated with the HRTF to generate the customized version of the HRTF, based on the at least one perceptive response from the user.

9. The method of claim 1 , wherein adjusting the at least one property of the HRTF further comprises:

interpolating at least one parameter associated with the HRTF across a plurality of clusters of a plurality of parameters associated with the HRTF to generate the customized version of the HRTF, based on the at least one perceptive response from the user.

10. The method of claim 1 , wherein adjusting the at least one property of the HRTF further comprises:

adjusting the one or more parameters associated with the HRTF using the ML model to generate the customized version of the HRTF, based on the at least one perceptive response from the user.

11. The method of claim 1 , wherein adjusting the at least one property of the HRTF further comprises:

dynamically adjusting at least one parameter associated with the HRTF by mapping the at least one parameter to a nonlinear statistical model to generate the customized version of the HRTF, based on the at least one perceptive response from the user.

12. The method of claim 1 , wherein adjusting the at least one property of the HRTF further comprises:

selecting, by the user via the interface device, a pair of elevation indications based on a pair of perceptive responses from the user; and

adjusting at least one parameter associated with the HRTF to generate the customized version of the HRTF, based on the selected pair of elevation indications.

13. The method of claim 1 , further comprising:

adjusting the at least one property of the HRTF to generate the customized version of the HRTF, further based on a movement of at least one of a head of the user and an eye gaze of the user responsive to the presentation of at least one spatialized version of the sound source.

14. The method of claim 1 , wherein adjusting the at least one property of the HRTF further comprises:

adjusting at least one interaural time difference (ITD) of the HRTF based on the at least one perceptive response from the user; and

interpolating one or more ITDs of the HRTF to generate the customized version of the HRTF, based on the at least one adjusted ITD.

15. The method of claim 1 , further comprising:

adjusting the initial version of the HRTF to obtain an adjusted version of the HRTF, based on a plurality of perceptive responses from the user when a plurality of audio signals are presented to the user originating from the sound source positioned at a plurality of locations in the local area; and

interpolating, using the adjusted version of the HRTF, at least one parameter associated with the HRTF corresponding to at least one additional location of the sound source in the local area to generate the customized version of the HRTF.

16. A non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

spatialize a sound source for an initial position in a local area using an initial version of a head-related transfer function (HRTF) to obtain an initial spatialized sound source;

upon presenting the initial spatialized sound source to a user, adjust, in an iterative manner based on at least one perceptive response from the user, at least one property of the HRTF to generate a version of the HRTF customized for the user, each perceptive response from the user indicating a respective offset between a perceived position and a target position of the sound source upon presentation of at least one spatialized version of the sound source, wherein adjusting the at least one property of the HRTF comprises:

pointing, by the user via an interface device, to at least one location in the local area as at least one perceived location of the sound source, the at least one location being outside of a field of view of the user, and

extrapolating, based on the at least one pointed location and using a machine learning model, one or more parameters associated with the HRTF to generate the customized version of the HRTF;

apply the customized version of the HRTF to one or more audio channels to form spatialized audio content for the perceived position; and

present the spatialized audio content to the user, wherein the offset between the perceived position and the target position is reduced.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions further cause the processor to:

spatialize the sound source for the initial position using an audio renderer, the audio renderer approximating head-related transfer functions (HRTFs) for the user, and the approximation is based on values of a plurality of parameters used by the audio renderer;

present the initial spatialized sound source using the values of the plurality of parameters;

adjust, in the iterative manner based on the at least one perceptive response from the user, the values of the plurality of parameters to reduce the offset;

spatialize the sound source for the perceived position using the audio renderer configured with the adjusted values of the plurality of parameters; and

present the sound source spatialized with the adjusted values, wherein the offset between the perceived position and the target position is reduced.

18. An audio system comprising:

an audio controller configured to:

spatialize a sound source for an initial position in a local area using an initial version of a head-related transfer function (HRTF) to obtain an initial spatialized sound source,

upon presenting the initial spatialized sound source to a user, adjust, in an iterative manner based on at least one perceptive response from the user, at least one property of the HRTF to generate a version of the HRTF customized for the user, each perceptive response from the user indicating a respective offset between a perceived position and a target position of the sound source upon presentation of at least one spatialized version of the sound source, wherein adjusting the at least one property of the HRTF comprises:

pointing, by the user via an interface device, to at least one location in the local area as at least one perceived location of the sound source, the at least one location being outside of a field of view of the user, and

extrapolating, based on the at least one pointed location and using a machine learning model, one or more parameters associated with the HRTF to generate the customized version of the HRTF, and

apply the customized version of the HRTF to one or more audio channels to form spatialized audio content for the perceived position; and

a transducer array coupled to the audio controller, the transducer array configured to present the spatialized audio content to the user, wherein the offset between the perceived position and the target position is reduced.

Assignments (3)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060314/0965 →
CORRECTIVE ASSIGNMENT TO CORRECT THE 3 AND 5 ASSIGNOR'S NAME PREVIOUSLY RECORDED AT REEL: 057817 FRAME: 0073. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Nov 15, 2021
From: BRIMIJOIN, WILLIAM OWEN, II; RUDZKI, TOMASZ; AMENGUAL GARÍ, SEBASTIÀ VICENÇ; WARNECKE, MICHAELA; FRANCL, ANDREW
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 058771/0924 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2021
From: BRIMIJOIN, WILLIAM OWEN, II; RUDZKI, TOMASZ; AMENGUAL GARI, SEBASTIÀ VICENÇ; WARNECKE, MICHAELA; FRANCL, ANDREW FREDERICK
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 057817/0073 →
Continuity (1)
Provisional Application 63158606 · Mar 9, 2021
Cited By (1)
US 12,340,631