Audio source localization system and method
View Patent ↗Systems and methods are described that perform audio source localization in a manner that provides increased robustness and responsiveness in the presence of acoustic echo. The systems and methods calculate a difference between a signal level associated with one or more of the audio signals generated by a microphone array and an estimated level of acoustic echo associated with one or more of the audio signals. This information is then used to determine whether and/or how to perform audio source localization. For example, a controller may use the difference to determine whether or not to freeze an audio source localization module that operates on the audio signals. As another example, the audio source localization module may incorporate the difference (or the estimated level of acoustic echo used to calculate the difference) into the logic that is used to determine the location of a desired audio source.
1. A method for performing audio source localization in a system comprising an array of microphones configured to generate a plurality of audio signals and an audio source localization module configured to process the plurality of audio signals to determine the location of a desired audio source, the method comprising:
calculating a difference between a signal level associated with one or more of the plurality of audio signals and an estimated level of acoustic echo associated with one or more of the plurality of audio signals; and
selectively disabling or enabling the audio source localization module based at least on the difference.
2. The method of claim 1 , further comprising:
determining the estimated level of acoustic echo associated with one or more of the plurality of audio signals by applying an estimated echo return loss to a level of an audio signal that is processed by the system for playback by one or more loudspeakers.
3. The method of claim 1 , wherein the system further comprises acoustic echo cancellers configured to apply acoustic echo cancellation to the plurality of audio signals prior to processing of the plurality of audio signals by the audio source localization module and wherein calculating the difference comprises:
calculating a difference between a signal level associated with one or more of the plurality of audio signals after application of acoustic echo cancellation thereto and an estimated level of residual acoustic echo associated with one or more of the plurality of the audio signals after application of acoustic echo cancellation thereto.
4. The method of claim 1 , wherein calculating the difference comprises calculating a difference for each audio signal in the plurality of audio signals between a signal level associated with the audio signal and a level of acoustic echo associated with the audio signal, and
wherein selectively disabling or enabling the audio source localization module based at least on the difference comprises selectively disabling or enabling the audio source localization module based at least on the difference calculated for each audio signal.
5. The method of claim 4 , wherein calculating the difference for each audio signal comprises calculating a difference for each of a plurality of frequency sub-bands for each audio signal between a signal level associated with the audio signal in the frequency sub-band and a level of acoustic echo associated with the audio signal in the frequency sub-band, and
wherein selectively disabling or enabling the audio source localization module based at least on the difference calculated for each audio signal comprises selectively disabling or enabling the audio source localization module based at least on the difference calculated for each frequency sub-band for each audio signal.
6. The method of claim 5 , wherein selectively disabling or enabling the audio source localization module based at least on the difference calculated for each frequency sub-band for each audio signal comprises:
identifying frequency sub-bands in which the difference exceeds a first threshold for every audio signal; and
selectively disabling or enabling the audio source localization module based at least on the identified frequency sub-bands.
7. The method of claim 6 , wherein selectively disabling or enabling the audio source localization module based at least on the identified frequency sub-bands comprises:
selectively disabling or enabling the audio source localization module based at least on whether the number of identified frequency sub-bands exceeds a second threshold.
8. The method of claim 7 , further comprising:
when the number of identified frequency sub-band exceeds the second threshold, enabling the audio source localization module to perform audio source localization by processing only components of the plurality of audio signals located in the identified frequency sub-bands to determine the location of the desired audio source.
9. A system, comprising:
an array of microphones that generates a plurality of audio signals;
an audio source localization module that processes the plurality of audio signals to determine the location of a desired audio source; and
a controller that calculates a difference between a signal level associated with one or more of the plurality of audio signals and an estimated level of acoustic echo associated with one or more of the plurality of audio signals and selectively disables or enables the audio source localization module based at least on the difference.
10. The system of claim 9 , further comprising:
a plurality of acoustic echo cancellers that apply acoustic echo cancellation to the plurality of audio signals prior to processing of the plurality of audio signals by the audio source localization module;
wherein the controller calculates the difference by calculating a difference between a signal level associated with one or more of the plurality of audio signals after application of acoustic echo cancellation thereto and an estimated level of residual acoustic echo associated with one or more of the plurality of audio signals after application of acoustic echo thereto.
11. The system of claim 9 , further comprising:
a location-based application that uses the determined location of the desired audio source from the audio source localization module to perform at least one operation.
12. The system of claim 10 , further comprising:
an output audio processing module configured to process and generate an output audio signal;
wherein the controller is configured to determine the estimated level of residual acoustic echo associated with one or more of the plurality of audio signals by
estimating an echo return loss (ERL) associated with the one or more of the plurality of audio signals and subtracting the ERL from a level of the output audio signal, or
estimating an ERL associated with the one or more of the plurality of audio signals and estimating an echo return loss enhancement (ERLE) and subtracting the estimated ERL and ERLE from a level of the output audio signal.
13. A computer program product comprising a computer-readable storage device having computer control logic recorded thereon that, when executed by one or more processors, causes the one or more processors to perform operations that include:
calculating a difference between a signal level associated with one or more of a plurality of audio signals generated by an array of microphones and an estimated level of acoustic echo associated with one or more of the plurality of audio signals; and
selectively disabling or enabling an audio source localization module based at least on the difference, the audio source localization module being configured to process the plurality of audio signals to determine the location of a desired audio source.
14. The computer program product of claim 13 , wherein the operations further include:
determining the estimated level of acoustic echo associated with one or more of the plurality of audio signals by applying an estimated echo return loss to a level of an audio signal that is processed for playback by one or more loudspeakers.
15. The computer program product of claim 13 , wherein the operations further include:
calculating a difference between a signal level associated with one or more of the plurality of audio signals after application of acoustic echo cancellation thereto and an estimated level of residual acoustic echo associated with one or more of the plurality of the audio signals after application of acoustic echo cancellation thereto.
16. The computer program product of claim 13 , wherein calculating the difference comprises calculating a difference for each audio signal in the plurality of audio signals between a signal level associated with the audio signal and a level of acoustic echo associated with the audio signal, and
wherein selectively disabling or enabling the audio source localization module based at least on the difference comprises selectively disabling or enabling the audio source localization module based at least on the difference calculated for each audio signal.
17. The computer program product of claim 16 , wherein calculating the difference for each audio signal comprises calculating a difference for each of a plurality of frequency sub-bands for each audio signal between a signal level associated with the audio signal in the frequency sub-band and a level of acoustic echo associated with the audio signal in the frequency sub-band, and
wherein selectively disabling or enabling the audio source localization module based at least on the difference calculated for each audio signal comprises selectively disabling or enabling the audio source localization module based at least on the difference calculated for each frequency sub-band for each audio signal.
18. The computer program product of claim 17 , wherein selectively disabling or enabling the audio source localization module based at least on the difference calculated for each frequency sub-band for each audio signal comprises:
identifying frequency sub-bands in which the difference exceeds a first threshold for every audio signal; and
selectively disabling or enabling the audio source localization module based at least on the identified frequency sub-bands.
19. The computer program product of claim 18 , wherein selectively disabling or enabling the audio source localization module based at least on the identified frequency sub-bands comprises:
selectively disabling or enabling the audio source localization module based at least on whether the number of identified frequency sub-bands exceeds a second threshold.
20. The computer program product of claim 19 , wherein the operations further include:
when the number of identified frequency sub-band exceeds the second threshold, enabling the audio source localization module to perform audio source localization by processing only components of the plurality of audio signals located in the identified frequency sub-bands to determine the location of the desired audio source.