Microphone noise suppression with beamforming
Techniques for improving microphone noise suppression are provided. A system for noise-suppression may include a beam selector component that applies logic to select a beam most likely corresponding to a direction of a noise source and keeps the beam selection steady rather than switching the beam too often to avoid processing complications. The selected beam may be used as a reference in an adaptive filter which outputs a noise estimate. The noise estimate and raw microphone data may be used to adapt the adaptive filter. A parallel filter which adapts after a time delay may be applied to the reference in order to prevent interference. An attenuation factor may be used to scale the noise estimate based on noise diffuseness, signal quality, and/or a gain limit. The scaled noise estimate may be subtracted from microphone input data to produce output audio data with improved signal quality and maintained signal coherence.
1. A computer-implemented method comprising:
receiving first audio data associated with a first microphone and second audio data associated with a second microphone, the first audio data and the second audio data associated with audio from a first time interval;
determining, using one or more beamformers and based at least in part on at least one of the first audio data or the second audio data, a first audio signal corresponding to a first direction and a second audio signal corresponding to a second direction;
determining that the first audio signal includes a first representation of first acoustic noise;
determining a first filter coefficient value using third audio data corresponding to a second time interval occurring prior to the first time interval;
processing the first audio signal using the first filter coefficient value to determine noise estimate data corresponding to a second representation of the first acoustic noise; and
determining output audio data based on at least in part on the first audio data, the second audio data, and the noise estimate data.
2. The computer-implemented method of claim 1 , further comprising:
determining a first time delay corresponding to an estimated length of time associated with utterance of a wakeword,
wherein processing the first audio signal using the first filter coefficient value to determine the noise estimate data is based at least in part on the first time delay.
3. The computer-implemented method of claim 1 , further comprising:
determining that the second audio signal includes a second representation of the first acoustic noise;
determining that a first portion of the first audio signal corresponds to a higher energy level than a second portion of the second audio signal; and
determining an updated first filter coefficient value based on the first audio signal.
4. The computer-implemented method of claim 1 , further comprising:
determining that the first representation of the first acoustic noise corresponds to a first energy level associated with the first time interval;
determining that the second audio signal includes a second representation of the first acoustic noise;
determining that the second representation of the first acoustic noise corresponds to a second energy level associated with first time interval;
determining that the second energy level is higher than the first energy level;
determining that a first beam corresponding to the first audio signal corresponds to a first direction adjacent to a second direction corresponding to a second beam corresponding to the second audio signal;
selecting the first audio signal, the first audio signal having been previously selected in association with the second time interval; and
determining the first filter coefficient value based on the first audio signal.
5. The computer-implemented method of claim 1 , wherein:
the noise estimate data comprises a first portion of noise estimate data corresponding to the first microphone and a second portion of noise estimate data corresponding to the second microphone; and
determining the output audio data comprises:
subtracting the first portion of noise estimate data from the first audio data to determine a first portion of the output audio data; and
subtracting the second portion of noise estimate data from the second audio data to determine a second portion of the output audio data.
6. The computer-implemented method of claim 1 , further comprising:
determining an attenuation factor based at least in part on a signal quality associated with at least the first audio data, a diffuseness associated with at least the first audio signal, and a gain limit value;
determining attenuated noise estimate data based on the noise estimate data and the attenuation factor; and
determining the output audio data further based at least in part on the attenuated noise estimate data.
7. The computer-implemented method of claim 1 , further comprising:
determining a first time delay corresponding to an estimated length of time associated with utterance of a wakeword;
determining that first microphone audio data received prior to detection of the wakeword is representative of noise;
determining that second microphone audio data received during the first time delay is representative of the wakeword; and
determining that third microphone audio data received after the first time delay is representative of noise.
8. The computer-implemented method of claim 1 , further comprising:
determining an attenuation factor based at least in part on the first audio data, the second audio data, the first audio signal, and the second audio signal;
determining attenuated noise estimate data based on the noise estimate data and the attenuation factor; and
determining the output audio data further based at least in part on the attenuated noise estimate data.
9. The computer-implemented method of claim 8 , wherein:
the attenuated noise estimate data comprises first attenuated noise estimate data corresponding to the first microphone and second attenuated noise estimate data corresponding to the second microphone; and
determining the output audio data comprises:
subtracting the first attenuated noise estimate data from the first audio data to determine a first portion of the output audio data; and
subtracting the second attenuated noise estimate data from the second audio data to determine a second portion of the output audio data.
10. A system comprising:
at least one processor; and
memory including instructions operable to be executed by the at least one processor to cause the system to:
receive first audio data associated with a first microphone and second audio data associated with a second microphone, the first audio data and the second audio data associated with audio from a first time interval;
determine, using one or more beamformers and based at least in part on at least one of the first audio data and the second audio data, a first audio signal corresponding to a first direction and a second audio signal corresponding to a second direction;
determine that the first audio signal includes a first representation of first acoustic noise;
determine a first filter coefficient value using third audio data corresponding to a second time interval occurring prior to the first time interval;
process the first audio signal using the first filter coefficient value to determine noise estimate data corresponding to a second representation of the first acoustic noise; and
determine output audio data based at least in part on the first audio data, the second audio data, and the noise estimate data.
11. The system of claim 10 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine a first time delay corresponding to an estimated length of time associated with utterance of a wakeword,
wherein processing the first audio signal using the first filter coefficient value to determine the noise estimate data is based at least in part on the first time delay.
12. The system of claim 10 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine that the second audio signal includes a second representation of the first acoustic noise;
determine that a first portion of the first audio signal corresponds to a higher energy level than a second portion of the second audio signal; and
determine an updated first filter coefficient value based on the first audio signal.
13. The system of claim 10 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine that the first representation of the first acoustic noise corresponds to a first energy level associated with the first time interval;
determine that the second audio signal includes a second representation of the first acoustic noise;
determine that the second representation of the first acoustic noise corresponds to a second energy level associated with first time interval;
determine that the second energy level is higher than the first energy level;
determine that a first beam corresponding to the first audio signal corresponds to a first direction adjacent to a second direction corresponding to a second beam corresponding to the second audio signal;
select the first audio signal, the first audio signal having been previously selected in association with the second time interval; and
determine the first filter coefficient value based on the first audio signal.
14. The system of claim 10 , wherein:
the noise estimate data comprises a first portion of noise estimate data corresponding to the first microphone and a second portion of noise estimate data corresponding to the second microphone; and
determining the output audio data comprises:
subtracting the first portion of noise estimate data from the first audio data to determine a first portion of the output audio data; and
subtracting the second portion of noise estimate data from the second audio data to determine a second portion of the output audio data.
15. The system of claim 10 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine an attenuation factor based at least in part on a signal quality associated with at least the first audio data, a diffuseness associated with at least the first audio signal, and a gain limit value;
determine attenuated noise estimate data based on the noise estimate data and the attenuation factor; and
determining the output audio data further based at least in part on the attenuated noise estimate data.
16. The system of claim 10 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine a first time delay corresponding to an estimated length of time associated with utterance of a wakeword;
determine that first microphone audio data received prior to detection of the wakeword is representative of noise;
determine that second microphone audio data received during the first time delay is representative of the wakeword; and
determine that third microphone audio data received after the first time delay is representative of noise.
17. The system of claim 10 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine an attenuation factor based at least in part on the first audio data, the second audio data, the first audio signal, and the second audio signal; and
determine attenuated noise estimate data based on the noise estimate data and the attenuation factor,
wherein determining the output audio data uses the attenuated noise estimate data.
18. The system of claim 17 , wherein:
the attenuated noise estimate data comprises first attenuated estimate data corresponding to the first microphone and second attenuated estimate data corresponding to the second microphone; and
determining the output audio data comprises:
subtracting the first attenuated estimate data from the first audio data to determine a first portion of the output audio data; and
subtracting the second attenuated estimate data from the second audio data to determine a second portion of the output audio data.
19. A computer-implemented method, the method comprising:
determining a first audio signal corresponding to a first direction and a second audio signal corresponding to a second direction;
determining that (i) the first audio signal includes a first representation of first acoustic noise, (ii) the second audio signal includes a second representation of the first acoustic noise, and (iii) a first portion of the first audio signal including the first representation of the first acoustic noise corresponds to a higher energy level than a second portion of the second audio signal including the second representation of the first acoustic noise;
determining a first filter coefficient value based on the first audio signal;
processing the first audio signal using the first filter coefficient value to determine noise estimate data;
determining attenuated noise estimate data based on the noise estimate data and an attenuation factor; and
determining output audio data based at least in part on the attenuated noise estimate data.
20. The computer-implemented method of claim 19 , further comprising:
determining the attenuation factor based at least in part on a signal quality associated with the first audio signal and the second audio signal, a, a diffuseness associated with the first audio signal and the second audio signal, and a gain limit.