IP Library Granted Patent US 10,957,299
Granted Patent B2
US 10,957,299 · App. 16/379,450 · Granted Mar 23, 2021

Acoustic transfer function personalization using sound scene analysis and beamforming

Inventors: Vladimir Tourbabin (Sammamish, WA); Jacob Ryan Donley (Kirkland, WA); Antonio John Miller (Woodinville, WA); Ravish Mehra (Tacoma, WA)
Assignee: Facebook Technologies, LLC
G10K11/1781H04R1/406H04R3/002G10K2210/30232
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,957,299
App. No.
16/379,450
Granted
Mar 23, 2021
Kind
B2
Abstract

An audio system for a wearable device dynamically updates acoustic transfer functions. The audio system is configured to estimate a direction of arrival (DoA) of each sound source detected by a microphone array relative to a position of the wearable device within a local area. The audio system may track the movement of each sound source. The audio system may form a beam in the direction of each sound source. The audio system may identify and classify each sound source based on the sound source properties. Based on the DoA estimates, the movement tracking, and the beamforming, the audio system generates or updates the acoustic transfer functions for the sound sources.

Claims (53)

1. A method comprising:

detecting, via a microphone array of a wearable device, sounds from one or more sound sources in a local area of the wearable device;

estimating acoustic transfer functions associated with the sounds;

estimating a direction of arrival (DoA) of a sound source in the one or more sound sources;

calculating a first confidence level for the direction of arrival estimate;

tracking a movement of the sound source;

calculating a second confidence level for the movement of the sound source; and

updating, in response to the movement of the sound source, the acoustic transfer functions based in part on the first confidence level and the second confidence level.

2. The method of claim 1 , further comprising classifying the sound source based on a classification library.

3. The method of claim 1 , further comprising isolating a signal from the sound source from other sound sources in the local area of the wearable device.

4. The method of claim 2 , further comprising calculating a third confidence level for the classifying, and a fourth confidence level for a beamforming process.

5. The method of claim 4 , further comprising updating the acoustic transfer functions based on at least one of the third confidence level or the fourth confidence level.

6. The method of claim 1 , wherein the tracking comprises:

storing values for the number and locations of the one or more sound sources over time; and

detecting a change in at least one of the number or the locations.

7. The method of claim 1 , further comprising:

updating sound filters based in part on the updated acoustic transfer functions; and

presenting audio content based on the updated sound filters.

8. An audio system comprising:

a microphone array configured to detect sounds from one or more sound sources in a local area of the audio system; and

a controller configured to:

detect, via a microphone array of a wearable device, sounds from one or more sound sources in a local area of the wearable device;

estimate acoustic transfer functions associated with the sounds;

estimate a direction of arrival (DoA) of a sound source in the one or more sound sources;

calculate a first confidence level for the direction of arrival estimate;

track a movement of the sound source;

calculate a second confidence level for the movement of the sound source; and

update, in response to the movement of the sound source, the acoustic transfer functions based in part on the first confidence level and the second confidence level.

9. The audio system of claim 8 , wherein the controller is further configured to classify the sound source based on a classification library.

10. The audio system of claim 8 , wherein the controller is further configured to isolate a signal from the sound source from other sound sources in the local area of the wearable device.

11. The audio system of claim 9 , wherein the controller is further configured to calculate third confidence level for the classifying, and a fourth confidence level for a beamforming process.

12. The audio system of claim 11 , wherein the controller is further configured to update the acoustic transfer functions based on at least one of the third confidence level or the fourth confidence level.

13. The audio system of claim 8 , wherein the tracking of the movement comprises:

storing values for the number and locations of the one or more sound sources over time; and

detecting a change in at least one of the number or the locations.

14. The audio system of claim 8 , wherein the controller is further configured to:

update sound filters based in part on the updated acoustic transfer functions; and

present audio content based on the updated sound filters.

15. The audio system of claim 8 , wherein the controller is further configured to generate a notification of the movement of the sound source.

16. A non-transitory computer-readable storage medium comprising instructions executable by a processor, the instructions when executed causing the processor to perform actions comprising:

detecting, via a microphone array of a wearable device, sounds from one or more sound sources in a local area of the wearable device;

estimating acoustic transfer functions associated with the sounds;

estimating a direction of arrival (DoA) of a sound source in the one or more sound sources;

calculating a first confidence level for the direction of arrival estimate;

tracking a movement of the sound source;

calculating a second confidence level for the movement of the sound source; and

updating, in response to the movement of the sound source, the acoustic transfer functions based in part on the first confidence level and the second confidence level.

17. The non-transitory computer-readable storage medium of claim 16 , the actions further comprising classifying the sound source based on a classification library.

18. The non-transitory computer-readable storage medium of claim 17 , the actions further comprising calculating a third confidence level for the classifying, and a fourth confidence level for a beamforming process.

19. The non-transitory computer-readable storage medium of claim 17 , the actions further comprising:

updating sound filters based in part on the updated acoustic transfer functions; and

presenting audio content based on the updated sound filters.

20. The non-transitory computer-readable storage medium of claim 16 , the actions further comprising isolating a signal from the sound source from other sound sources in the local area of the wearable device.

Assignments (2)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060315/0224 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 22, 2019
From: TOURBABIN, VLADIMIR; DONLEY, JACOB RYAN; MILLER, ANTONIO JOHN; MEHRA, RAVISH
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 048959/0029 →
Continuity (1)
Related Publication 20200327877A1 · Oct 15, 2020
Cited By (1)
US 12,713,183