IP Library Granted Patent US 12,219,331
Granted Patent B2
US 12,219,331 · App. 18/384,243 · Granted Feb 4, 2025

Audio system for suppressing leakage signal from in-ear device

Inventors: William Owen Brimijoin, II (Kirkland, WA); Nils Thomas Fritiof Lunner (Redmond, WA); Philip Robinson (Seattle, WA); Ravish Mehra (Seattle, WA)
Assignee: Meta Platforms Technologies, LLC
H04R3/04G02B27/0172G06F3/013G06F3/017G06T19/006H04R1/1016H04R5/033H04R5/04H04S1/005H04S7/304G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,219,331
App. No.
18/384,243
Granted
Feb 4, 2025
Kind
B2
Abstract

A system for presenting audio content to a user. The system comprises one or more microphones coupled to a frame of a headset. The one or more microphones capture sound from a local area. The system further comprises an audio controller integrated into the headset and communicatively coupled to an in-ear device worn by a user. The audio controller identifies one or more sound sources in the local area based on the captured sound. The audio controller further determines a target sound source of the one or more sound sources and determines one or more filters to apply to a sound signal associated with the target sound source in the captured sound. The audio controller further generates an augmented sound signal by applying the one or more filters to the sound signal and provides the augmented sound signal to the in-ear device for presentation to a user.

Claims (68)

1. An audio system comprising:

a head-mounted device worn by a user;

one or more microphones configured to capture sounds from a local area, the captured sounds including a leakage signal from the head-mounted device;

an audio controller configured to determine a mitigation signal that is out-of-phase to the leakage signal; and

a transducer array configured to provide the mitigation signal as an output signal, wherein the mitigation signal suppresses the leakage signal.

2. The audio system of claim 1 , wherein the leakage signal is a sound signal that is in-phase with an audio signal presented as audio content to the user via the head-mounted device.

3. The audio system of claim 1 , wherein the mitigation signal suppresses the leakage signal in a far field.

4. The audio system of claim 1 , wherein the audio controller is further configured to:

generate, based at least in part on the leakage signal, a filter for filtering out frequencies of the leakage signal;

apply the generated filter to the leakage signal to generate the mitigation signal; and

provide the mitigation signal to the transducer array.

5. The audio system of claim 1 , wherein the audio controller is further configured to:

detect that a target sound source is within a threshold distance of a different sound source in the local area, wherein the mitigation signal suppresses the leakage signal received from the different sound source; and

update one or more filters to increase spatial separation between the target sound source and the different sound source to be at least a separation distance that is greater than the threshold distance.

6. The audio system of claim 5 , wherein the audio controller is further configured to:

generate an augmented sound signal by applying the updated one or more filters to a sound signal from the local area; and

provide the augmented sound signal to the transducer array for presentation to the user.

7. The audio system of claim 5 , wherein the audio controller is further configured to:

determine the one or more filters based at least in part on one or more of: a location of the user in the local area, a location of each of one or more sound sources in the local area, one or more acoustic parameters of the local area, and types of sound sources in the local area.

8. The audio system of claim 5 , wherein the head-mounted device is configured to:

receive the updated one or more filters from the audio controller;

generate an augmented sound signal by applying the updated one or more filters to a sound signal from the local area; and

present the augmented sound signal to the user via a transducer of the head-mounted device.

9. The audio system of claim 5 , wherein the audio controller is further configured to determine the target sound source based at least in part on a gaze of the user.

10. The audio system of claim 5 , wherein the audio controller is further configured to determine the target sound source based at least in part on feedback provided by the user.

11. The audio system of claim 5 , wherein the audio controller is further configured to:

receive feedback information about a gesture performed by the user;

determine, based on the feedback information, that the gesture corresponds to a selection of a sound source; and

identify the selected sound source as the target sound source.

12. The audio system of claim 1 , wherein:

the audio controller is further configured to determine one or more filters based at least in part on a sound profile of the user; and

the head-mounted device is configured to:

receive the one or more filters from the audio controller,

generate an augmented sound signal by applying the one or more filters to a sound signal from a target sound source in the local area; and

present the augmented sound signal to the user via a transducer of the head-mounted device.

13. The audio system of claim 1 , wherein:

the audio controller is further configured to select, based at least in part on a sound profile of the user, a wide dynamic range compression filter from a plurality of wide dynamic range compression filters; and

the head-mounted device is configured to:

receive the selected wide dynamic range compression filter from the audio controller,

generate an augmented sound signal by applying the selected wide dynamic range compression filter to a sound signal from a target sound source in the local area; and

present the augmented sound signal to the user via a transducer of the head-mounted device.

14. A method performed by an audio system, the method comprising:

capturing, by one or more microphones of the audio system, sounds from a local area, the captured sounds including a leakage signal from an head-mounted device of the audio system worn by a user;

determining, by an audio controller of the audio system, a mitigation signal that is out-of-phase to the leakage signal; and

providing, by a transducer array of the audio system, the mitigation signal as an output signal, wherein the mitigation signal suppresses the leakage signal.

15. The method of claim 14 , further comprising:

detecting, by the audio controller, that a target sound source is within a threshold distance of a different sound source in the local area, wherein the mitigation signal suppresses the leakage signal received from the different sound source;

updating, by the audio controller, one or more filters to increase spatial separation between the target sound source and the different sound source to be at least a separation distance that is greater than the threshold distance;

generating, by the head-mounted device, an augmented sound signal by applying the updated one or more filters to a sound signal from the local area; and

presenting, via a transducer of the head-mounted device, the augmented sound signal to the user.

16. The method of claim 15 , further comprising:

determining, by the audio controller, the one or more filters based at least in part on one or more of: a location of the user in the local area, a location of each of one or more sound sources in the local area, one or more acoustic parameters of the local area, and types of sound sources in the local area.

17. The method of claim 15 , further comprising:

determining, by the audio controller, the target sound source based at least in part on a gaze of the user.

18. The method of claim 14 , further comprising:

determining, by the audio controller, one or more filters based at least in part on a sound profile of the user; and

generating, by the head-mounted device, an augmented sound signal by applying the one or more filters to a sound signal from a target sound source in the local area; and

presenting the augmented sound signal to the user via a transducer of the head-mounted device.

19. A headset comprising:

one or more microphones coupled to a frame of the headset worn by a user, the one or more microphones configured to capture sounds from a local area, the captured sounds including a leakage signal from a head-mounted device worn by the user that is separate from and communicatively coupled to the headset;

an audio controller integrated into the headset and communicatively coupled to the head-mounted device, the audio controller configured to determine a mitigation signal that is out-of-phase to the leakage signal; and

a transducer array configured to provide the mitigation signal as an output signal, wherein the mitigation signal suppresses the leakage signal.

20. The headset of claim 19 , wherein the audio controller is further configured to:

determine one or more filters based at least in part on one or more of: a location of the user in the local area, a location of each of one or more sound sources in the local area, one or more acoustic parameters of the local area, and types of sound sources in the local area;

detect that a target sound source is within a threshold distance of a different sound source in the local area, wherein the mitigation signal suppresses the leakage signal received from the different sound source;

update the one or more filters to increase spatial separation between the target sound source and the different sound source to be at least a separation distance that is greater than the threshold distance;

generate an augmented sound signal by applying the updated one or more filters to a sound signal from the local area; and

provide the augmented sound signal to the transducer array for presentation to the user.

Continuity (4)
Continuation 17557393 · Dec 21, 2021
Continuation 16945388 · Jul 31, 2020
Provisional Application 63052016 · Jul 15, 2020
Related Publication 20240056733A1 · Feb 15, 2024
References Cited (26)
US 10638252B1 · Donley et al. · 2020 [cited by applicant]
US 10812896B2 · Oishi et al. · 2020 [cited by applicant]
US 10869120B1 · Jacobs et al. · 2020 [cited by applicant]
US 10893357B1 · Oishi et al. · 2021 [cited by applicant]
US 11134349B1 · Nagar et al. · 2021 [cited by applicant]
US 11202145B1 · Oishi et al. · 2021 [cited by applicant]
US 11245984B1 · Brimijoin, II · 2022 [cited by examiner]
US 11843926B2 · Brimijoin, II · 2023 [cited by examiner]
US 20050195988A1 · Tashev et al. · 2005 [cited by applicant]
US 20150156592A1 · Jensen · 2015 [cited by applicant]
US 20150358732A1 · Yu et al. · 2015 [cited by applicant]
US 20160080874A1 · Fullam · 2016 [cited by applicant]
US 20160313973A1 · Yajima · 2016 [cited by applicant]
US 20170353793A1 · Sun et al. · 2017 [cited by applicant]
US 20190238971A1 · Wakeland et al. · 2019 [cited by applicant]
US 20190394576A1 · Petersen et al. · 2019 [cited by applicant]
CN 110473567A · 2019 [cited by applicant]
Co-Pending U.S. Appl. No. 16/555,058, filed Aug. 29, 2019, 58 Pages. [cited by applicant]
Co-Pending U.S. Appl. No. 16/737,316, filed Jan. 8, 2020, 47 Pages. [cited by applicant]
Co-Pending U.S. Appl. No. 16/790,613, filed Feb. 13, 2020, 47 Pages. [cited by applicant]
European Search Report for European Patent Application No. 23176818.5, dated Aug. 29, 2023, 07 pages. [cited by applicant]
European Search Report for European Patent Application No. 23176819.3, dated Aug. 25, 2023, 08 pages. [cited by applicant]
International Preliminary Report on Patentability for International Application No. PCT/US2021/036053, mailed Jan. 26, 2023, 18 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2021/036053, mailed Nov. 25, 2021, 21 pages. [cited by applicant]
Invitation to Pay Additional Fees received for International Application No. PCT/US2021/036053, mailed on Sep. 30, 2021, 16 pages. [cited by applicant]
Souza P. E., “Effects of Compression on Speech Acoustics, Intelligibility, and Sound Quality,” Trends in Amplification, Dec. 2002, vol. 6 (4), pp. 131-165. [cited by applicant]