IP Library Granted Patent US 11,330,371
Granted Patent B2
US 11,330,371 · App. 17/079,625 · Granted May 10, 2022

Audio control based on room correction and head related transfer function

Inventors: Gregory Carlsson (San Diego, CA); James R. Milne (San Diego, CA)
Assignee: SONY GROUP CORPORATION
H04R3/04H04S7/306H04S7/307H04R2430/20H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,330,371
App. No.
17/079,625
Granted
May 10, 2022
Kind
B2
Abstract

An audio reproduction device and method for audio control based on room-correction (RC) and head related transfer function (HRTF) are provided. The audio reproduction device includes a speaker that reproduces a first audio signal. The audio reproduction device receives a plurality of second audio signals indicative of frequency responses captured based on the first audio signal and captured by a plurality of audio capturing devices positioned on a head wearable device of a user present within an enclosed physical space. The audio reproduction device determines RC preset for one or more RC filters associated with the speaker, based on the captured frequency responses. The audio reproduction device further determines HRTF associated with the user based on the captured frequency responses, and user-specific information of the user. The audio reproduction device further controls audio reproduction of the speaker based on the determined RC preset and the determined HRTF.

Claims (103)

1. An audio reproduction device, comprising:

a speaker configured to reproduce a first audio signal;

circuitry coupled with the speaker, wherein the circuitry is configured to:

receive a plurality of second audio signals captured by a plurality of audio capturing devices, wherein

the plurality of audio capturing devices is on a head wearable device of a first user present at a specific location within an enclosed physical space,

the plurality of second audio signals is received at the specific location of the first user,

each second audio signal of the plurality of second audio signals indicates a respective frequency response of a plurality of frequency responses of the plurality of second audio signals,

the plurality of frequency responses corresponds to the specific location of the first user within the enclosed physical space, and

the plurality of frequency responses is captured based on the first audio signal reproduced by the speaker;

determine an average value of the plurality of frequency responses indicated by the received plurality of second audio signals;

determine a room-correction (RC) preset for at least one RC filter associated with the speaker, based on the determined average value of the plurality of frequency responses indicated by the received plurality of second audio signals,

wherein the determined RC preset corresponds to the specific location of the first user within the enclosed physical space;

determine a first head related transfer function (HRTF) associated with the first user based on:

the plurality of frequency responses indicated by the received plurality of second audio signals, and

user-specific information corresponding to the first user,

wherein the first HRTF is determined for at least one HRTF filter associated with the speaker; and

control audio reproduction of the speaker based on:

the determined RC preset corresponding to the specific location within the enclosed physical space, and

the determined first HRTF corresponding to the first user present within the enclosed physical space.

2. The audio reproduction device according to claim 1 , wherein the RC preset comprises at least one filter coefficient associated with the at least one RC filter of the speaker.

3. The audio reproduction device according to claim 1 , further comprising a memory configured to:

store the determined RC preset corresponding to the specific location within the enclosed physical space, and

store the determined first HRTF corresponding to the first user present within the enclosed physical space.

4. The audio reproduction device according to claim 1 , wherein the user-specific information comprises at least one of dimensions of a head of the first user, dimensions of ears of the first user, dimensions of ear canals of the first user, dimensions of a shoulder of the first user, dimensions of a torso of the first user, a density of the head of the first user, or an orientation of the head of the first user.

5. The audio reproduction device according to claim 1 , wherein the circuitry is further configured to:

determine an interaural time difference (ITD) and an interaural level difference (ILD) for the first user based on the plurality of frequency responses indicated by the received plurality of second audio signals; and

determine the first HRTF associated with the first user based on the determined ITD and the determined ILD.

6. The audio reproduction device according to claim 1 , wherein the circuitry is further configured to:

determine a first value of at least one coefficient of the at least one HRTF filter of the speaker;

determine a second value of at least one coefficient of the at least one RC filter of the speaker; and

control the audio reproduction of the speaker based on the determined first value of the at least one coefficient of the at least one HRTF filter and the determined second value of the at least one coefficient of the at least one RC filter.

7. The audio reproduction device according to claim 6 , further comprising an Input-Output (I/O) interface, wherein the circuitry is further configured to:

receive a user input via the I/O interface; and

determine each of the first value of the at least one coefficient of the at least one HRTF filter and the second value of the at least one coefficient of the at least one RC filter, based on the received user input.

8. The audio reproduction device according to claim 1 , wherein

the circuitry is further configured to receive occupancy information from at least one sensor communicably coupled to the audio reproduction device,

the occupancy information indicates a number of users of a set of users present within the enclosed physical space, and

the set of users includes the first user.

9. The audio reproduction device according to claim 8 , wherein the circuitry is further configured to:

determine whether a second HRTF is calibrated for a second user of the set of users, wherein the second user is different from the first user; and

determine a first value of at least one coefficient of the at least one HRTF filter and a second value of at least one coefficient of the at least one RC filter based on the determination of the calibration of the second HRTF for the second user.

10. The audio reproduction device according to claim 9 , wherein the circuitry is further configured to:

set the first value of the at least one coefficient of the at least one HRTF filter and the second value of the at least one coefficient of the at least one RC filter, based on the received occupancy information, wherein

the received occupancy information which indicates the number of users as more than one, and

the second value is set higher than the first value; and

control the audio reproduction of the speaker based on the set first value of the at least one coefficient of the at least one HRTF filter and the set second value of the at least one coefficient of the at least one RC filter.

11. A method, comprising:

in an audio reproduction device, which includes a speaker configured to reproduce a first audio signal:

receiving a plurality of second audio signals captured by a plurality of audio capturing devices, wherein

the plurality of audio capturing devices is on a head wearable device of a first user present at a specific location within an enclosed physical space, wherein

the plurality of second audio signals is received at the specific location of the first user,

each second audio signal of the plurality of second audio signals indicates a respective frequency response of a plurality of frequency responses of the plurality of second audio signals,

the plurality of frequency responses corresponds to the specific location of the first user within the enclosed physical space, and

the plurality of frequency responses is captured based on the first audio signal reproduced by the speaker;

determining an average value of the plurality of frequency responses indicated by the received plurality of second audio signals;

determining a room-correction (RC) preset for at least one RC filter associated with the speaker, based on the determined average value of the plurality of frequency responses indicated by the received plurality of second audio signals,

wherein the determined RC preset corresponds to the specific location of the first user within the enclosed physical space;

determining a first head related transfer function (HRTF) associated with the first user based on:

the plurality of frequency responses indicated by the received plurality of second audio signals, and

user-specific information corresponding to the first user,

wherein the first HRTF is determined for at least one HRTF filter associated with the speaker; and

controlling audio reproduction of the speaker based on:

the determined RC preset corresponding to the specific location within the enclosed physical space, and

the determined first HRTF corresponding to the first user present within the enclosed physical space.

12. The method according to claim 11 , further comprising:

determining an interaural time difference (ITD) and an interaural level difference (ILD) for the first user based on the plurality of frequency responses indicated by the received plurality of second audio signals; and

determining the first HRTF associated with the first user based on the determined ITD and the determined ILD.

13. The method according to claim 11 , further comprising:

determining a first value of at least one coefficient of the at least one HRTF filter of the speaker;

determining a second value of at least one coefficient of the at least one RC filter of the speaker; and

controlling the audio reproduction of the speaker based on the determined first value of the at least one coefficient of the at least one HRTF filter and the determined second value of the at least one coefficient of the at least one RC filter.

14. The method according to claim 13 , further comprising:

receiving a user input, via an Input-Output (I/O) interface of the audio reproduction device; and

determining each of the first value of the at least one coefficient of the at least one HRTF filter and the second value of the at least one coefficient of the at least one RC filter, based on the received user input.

15. The method according to claim 11 , further comprising receiving occupancy information from at least one sensor communicably coupled to the audio reproduction device, wherein

the occupancy information indicates a number of users of a set of users present within the enclosed physical space, and

the set of users includes the first user.

16. The method according to claim 15 , further comprising:

determining whether a second HRTF is calibrated for a second user of the set of users, wherein the second user is different from the first user; and

determining a first value of at least one coefficient of the at least one HRTF filter and a second value of at least one coefficient of the at least one RC filter based on the determination of the calibration of the second HRTF for the second user.

17. The method according to claim 16 , further comprising:

setting the first value of the at least one coefficient of the at least one HRTF filter and the second value of the at least one coefficient of the at least one RC filter, based on the received occupancy information, wherein

the received occupancy information indicates the number of users as more than one, and

the second value is set higher than the first value; and

controlling the audio reproduction of the speaker based on the set first value of the at least one coefficient of the at least one HRTF filter and the set second value of the at least one coefficient of the at least one RC filter.

18. A non-transitory computer-readable medium having stored thereon, computer-executable instructions that when executed by an audio reproduction device, causes the audio reproduction device to execute operations, the operations comprising:

receiving a plurality of second audio signals captured by a plurality of audio capturing devices, wherein

the plurality of audio capturing devices is on a head wearable device of a first user present at a specific location within an enclosed physical space,

the plurality of second audio signals is received at the specific location of the first user,

each second audio signal of the plurality of second audio signals indicates a respective frequency response of a plurality of frequency responses of the plurality of second audio signals,

the plurality of frequency responses corresponds to the specific location of the first user within the enclosed physical space,

the plurality of frequency responses is captured based on a first audio signal, and

the first audio signal is reproduced by a speaker in the audio reproduction device;

determining an average value of the plurality of frequency responses indicated by the received plurality of second audio signals;

determining a room-correction (RC) preset for at least one RC filter associated with the speaker, based on the determined average value of the plurality of frequency responses indicated by the received plurality of second audio signals,

wherein the determined RC preset corresponds to the specific location of the first user within the enclosed physical space;

determining a first head related transfer function (HRTF) associated with the first user based on:

the plurality of frequency responses indicated by the received plurality of second audio signals, and

user-specific information corresponding to the first user,

wherein the first HRTF is determined for at least one HRTF filter associated with the speaker; and

controlling audio reproduction of the speaker based on:

the determined RC preset corresponding to the specific location within the enclosed physical space, and

the determined first HRTF corresponding to the first user present within the enclosed physical space.

Assignments (2)
CHANGE OF NAME Recorded Jun 29, 2021
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 056715/0919 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2020
From: CARLSSON, GREGORY; MILNE, JAMES R
To: SONY CORPORATION
Reel/Frame 054380/0956 →
Continuity (2)
Provisional Application 62931946 · Nov 7, 2019
Related Publication 20210144475A1 · May 13, 2021
Cited By (1)
US 12,445,795