IP Library Granted Patent US 11,425,523
Granted Patent B2
US 11,425,523 · App. 16/846,068 · Granted Aug 23, 2022

Systems and methods for audio adjustment

Inventor: Scott Phillip Selfon (Kirkland, WA)
Assignee: FACEBOOK TECHNOLOGIES, LLC
H04S7/303G10L13/02H04S7/307
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,425,523
App. No.
16/846,068
Granted
Aug 23, 2022
Kind
B2
Abstract

An audio system includes a sound output device, a microphone, and processing circuitry. The microphone is configured to capture environmental audio. The processing circuitry is configured to analyze the environmental audio to identify one or more properties of environmental audio conditions. The processing circuitry is configured to adjust one or more sound presentation parameters based on the one or more properties of the environmental audio conditions to account for the environmental audio conditions. The processing circuitry is configured to operate the sound output device to output audio according to the one or more sound presentation parameters.

Claims (59)

1. An audio system comprising:

a sound output device;

a first microphone and a second microphone configured to capture environmental audio data; and

processing circuitry configured to:

obtain environmental audio data from the first microphone and the second microphone of the audio system;

determine an arrival direction of environmental audio relative to the audio system based on a comparison between the environmental audio data obtained from the first microphone and the environmental audio data obtained from the second microphone;

adjust a virtual spatial position of a spatial audio simulation based on the arrival direction of the environmental audio, wherein the spatial audio simulation comprises simulating a sound at the virtual spatial position relative to the audio device to generate an audio output; and

provide the audio output to a user using the sound output device and provide an aural notification to the user that the virtual spatial position is adjusted.

2. The audio system of claim 1 , wherein the processing circuitry is further configured to analyze the environmental audio data to identify one or more properties of environmental audio conditions, wherein the one or more properties of environmental audio conditions comprise at least one of:

an amplitude of the environmental audio; or

an amplitude of the environmental audio within one or more particular frequency ranges.

3. The audio system of claim 2 , wherein the particular frequency range includes a frequency of the adjusted output audio of the sound output device.

4. The audio system of claim 1 , further comprising adjusting one or more sound presentation parameters of the sound output device, wherein the sound presentation parameters include any of a direction of arrival, a speech delivery style, an amplitude, or an amplitude across one or more frequency ranges of the output audio.

5. The audio system of claim 2 , further comprising a display screen configured to provide visual data to a user of the audio system, wherein the processing circuitry is configured to:

operate the display screen to provide the adjusted audio output of the sound output device as visual data in response to at least one of the one or more properties of the environmental audio conditions.

6. The audio system of claim 1 , wherein the processing circuitry is configured to:

transition the audio system between operating according to a plurality of modes in response to one or more properties of environmental audio conditions, wherein the sound output device of the audio system is configured to use a different set of audio presentation parameters for each of the plurality of modes;

wherein the audio system is configured to operate in a first mode of operation in response to a detected volume of speech of the user;

wherein the audio system is configured to operate in a second mode of operation in response to a background noise level exceeding a first threshold; and

wherein the audio system is configured to operate in a third mode of operation in response to the background noise level exceeding a second threshold greater than the first threshold, wherein sound output of the sound output device when operating in the third mode of operation is greater than sound output of the sound output device when operating in the second mode of operation.

7. The audio system of claim 1 , wherein the processing circuitry is configured to continuously adjust the virtual spatial position of the spatial audio simulation to maintain a degree of separation between an arrival direction of the simulated sound and the arrival direction of the environmental audio.

8. The audio system of claim 1 , wherein the environmental audio data obtained from the first microphone and the second microphone is obtained in real-time.

9. A method for adjusting audio output, the method comprising:

obtaining environmental audio from a first microphone of an audio device and a second microphone of the audio device, the first microphone positioned a distance from the second microphone;

comparing the environmental audio obtained from the first microphone to the environmental audio obtained from the second microphone to determine an arrival direction of the environmental audio relative to the audio device;

analyzing the environmental audio to identify one or more properties of environmental audio conditions, the one or more properties of the environmental audio conditions comprising at least one of an amplitude of the environmental audio, the amplitude of the environmental audio within a particular frequency range, or the arrival direction of the environmental audio relative to the audio device;

adjusting a speech synthesizer based at least on the one or more properties of the environmental audio conditions to generate an adjusted audio output for a sound output device that accounts for the environmental audio conditions; and

operating the sound output device to output the adjusted audio output.

10. The method of claim 9 , further comprising:

performing a simulation of a virtual spatial position from which a sound originates relative to the audio device to generate the audio output; and

adjusting the virtual spatial position from which the audio output originates based on the arrival direction of the environmental audio relative to the audio device.

11. The method of claim 10 , further comprising providing an aural notification to a user that the virtual spatial position is adjusted.

12. The method of claim 9 , further comprising:

transitioning the audio device between operating according to a plurality of modes in response to the one or more properties of environmental audio conditions, wherein the sound output device of the audio device is configured to use a different set of audio presentation parameters for each of the plurality of modes;

wherein the audio device is configured to operate in a first mode of operation in response to a detected volume of speech of the user;

wherein the audio device is configured to operate in a second mode of operation in response to a background noise level exceeding a first threshold; and

wherein the audio device is configured to operate in a third mode of operation in response to the background noise level exceeding a second threshold greater than the first threshold, wherein sound output of the sound output device when operating in the third mode of operation is greater than sound output of the sound output device when operating in the second mode of operation.

13. The method of claim 9 , further comprising continuously adjusting a virtual spatial position of a spatial audio simulation to maintain a degree of separation between an arrival direction of a simulated sound and the arrival direction of the environmental audio.

14. A method for adjusting audio output, the method comprising:

obtaining environmental audio data from a first microphone and a second microphone of an audio device;

determining an arrival direction of environmental audio relative to the audio device based on a comparison between the environmental audio data obtained from the first microphone and the environmental audio data obtained from the second microphone;

adjusting a virtual spatial position of a spatial audio simulation based on the arrival direction of the environmental audio, wherein the spatial audio simulation comprises simulating a sound at the virtual spatial position relative to the audio device to generate an audio output; and

providing the audio output to a user and providing an aural notification to the user that the virtual spatial position is adjusted.

15. The method of claim 14 , wherein the environmental audio data from the first microphone and the environmental audio data from the second microphone are obtained in real-time.

16. The method of claim 14 , further comprising:

determining an amplitude of the environmental audio based on at least one of the environmental audio data obtained from the first microphone or the environmental audio data obtained from the second microphone;

determining an amplitude of the environmental audio that is within a particular frequency range based on at least one of the environmental audio data obtained from the first microphone or the environmental audio data obtained from the second microphone; and

adjusting the audio output provided to the user based on at least one of the amplitude of the environmental audio or the amplitude of the environmental audio that is within the particular frequency range.

17. The method of claim 16 , wherein adjusting the audio output comprises at least one of:

adjusting an amplitude of the audio output;

adjusting a frequency or pitch of the audio output; or

adjusting an amplitude of the audio output across a frequency range.

18. The method of claim 14 , wherein the virtual spatial position of the spatial audio simulation is adjusted to maintain a separation between the virtual spatial position and the arrival direction of the environmental audio.

19. The method of claim 14 , further comprising:

transitioning the audio device between operating according to a plurality of modes in response to the one or more properties of environmental audio conditions, wherein a sound output device of the audio device is configured to use a different set of audio presentation parameters for each of the plurality of modes;

wherein the audio device is configured to operate in a first mode of operation in response to a detected volume of speech of the user;

wherein the audio device is configured to operate in a second mode of operation in response to a background noise level exceeding a first threshold; and

wherein the audio device is configured to operate in a third mode of operation in response to the background noise level exceeding a second threshold greater than the first threshold, wherein sound output of the sound output device when operating in the third mode of operation is greater than sound output of the sound output device when operating in the second mode of operation.

20. The method of claim 14 , further comprising continuously adjusting the virtual spatial position of the spatial audio simulation to maintain a degree of separation between an arrival direction of the simulated sound and the arrival direction of the environmental audio.

Assignments (2)
CHANGE OF NAME Recorded Jul 22, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060816/0634 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 24, 2020
From: SELFON, SCOTT PHILLIP
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 053307/0735 →