Reference free acoustic echo cancellation
A multi-microphone device that can perform acoustic echo cancellation (AEC) without an external reference signal. The device uses the audio data from one of its microphones as a reference for purposes of AEC and acoustic noise cancellation (ANC). The device determines filter coefficients for an adaptive filter for ANC when cancelling one microphone signal from another microphone's signal. Those filter coefficients are buffered and delayed and then used for AEC operations cancelling one microphone signal from another microphone's signal. When desired audio (such as a wakeword, speech, or the like) is detected, the device may freeze the coefficients for purposes of performing AEC until the desired audio is complete. The device may then continue adapting and using the coefficients.
1. A computer-implemented method comprising:
receiving, from a first microphone of a device, first audio data corresponding to a first time period;
receiving, from a second microphone of the device, second audio data corresponding to the first time period;
processing the first audio data and the second audio data using an adaptive filter to determine a first coefficient value corresponding to the first time period;
determining a delay time period corresponding to use of the first coefficient value;
receiving, from the first microphone, third audio data corresponding to a second time period, the second time period being the delay time period after the first time period;
receiving, from the second microphone, fourth audio data corresponding to the second time period; and
processing the third audio data, the fourth audio data, and the first coefficient value to determine cancelled data.
2. The computer-implemented method of claim 1 , wherein determination of the cancelled data comprises:
applying the first coefficient value to the fourth audio data to determine modified audio data; and
subtracting the modified audio data from the third audio data to determine the cancelled data.
3. The computer-implemented method of claim 1 , further comprising:
processing the cancelled data using a wakeword detection component to determine a representation of a wakeword; and
in response to determining the representation of the wakeword, performing speech processing based at least in part on the cancelled data.
4. The computer-implemented method of claim 1 , further comprising:
determining the cancelled data represents speech; and
in response to determining the cancelled data represents speech, continue using the first coefficient value for further audio data corresponding to a third time period after the second time period.
5. The computer-implemented method of claim 4 , further comprising:
determining the speech has ended; and
in response to determining the speech has ended, using a second coefficient value for echo cancellation, the second coefficient value corresponding to a fourth time period after the second time period.
6. The computer-implemented method of claim 1 , wherein processing the third audio data, the fourth audio data, and the first coefficient value to determine the cancelled data corresponds to a first mode of operation and wherein the method further comprises:
determining a loudspeaker in an environment of the device is emitting audio; and
in response to the loudspeaker emitting audio, entering the first mode of operation.
7. The computer-implemented method of claim 1 , wherein:
the third audio data includes a first representation of noise detected in an environment of the device;
the fourth audio data includes a second representation of the noise; and
determination of the cancelled data results in at least partial cancellation of the noise as represented in the cancelled data.
8. The computer-implemented method of claim 1 , further comprising:
emitting audio by a loudspeaker;
receiving, from at least one microphone of the device, fifth audio data including a representation of the audio;
processing the fifth audio data to determine acoustic characteristic data corresponding to an environment of the device; and
processing the acoustic characteristic data to determine the delay time period.
9. The computer-implemented method of claim 1 , further comprising:
receiving fifth audio data from a third microphone of the device;
processing at least the fourth audio data and the fifth audio data to determine a first matrix represent audio captured by the second microphone and the third microphone; and
determining a second matrix including at least the first coefficient value,
wherein determination of the cancelled data comprises processing the first matrix with respect to the second matrix to determine the cancelled data.
10. The computer-implemented method of claim 1 , further comprising:
determining the cancelled data represents an acoustic event; and
in response to determining the cancelled data represents the acoustic event, continue using the first coefficient value for further audio data corresponding to a third time period after the second time period.
11. A system comprising:
a first microphone of a device;
a second microphone of the device;
at least one processor; and
at least one memory including instructions operable to be executed by the at least one processor to cause the system to:
receive, from the first microphone, a first audio data corresponding to a first time period;
receive, from the second microphone, a second audio data corresponding to the first time period;
process the first audio data and the second audio data using an adaptive filter to determine a first coefficient value corresponding to the first time period;
determine a delay time period corresponding to use of the first coefficient value;
receive, from the first microphone, a third audio data corresponding to a second time period, the second time period corresponding to the delay time period after the first time period;
receive, from the second microphone, a fourth audio data corresponding to the second time period; and
process the third audio data, the fourth audio data, and the first coefficient value to determine cancelled data.
12. The system of claim 11 , wherein the instructions that cause the system to determine the cancelled data comprise instructions that, when executed by the at least one processor, cause the system to:
apply the first coefficient value to the fourth audio data to determine modified audio data; and
subtract the modified audio data from the third audio data to determine the cancelled data.
13. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
process the cancelled data using a wakeword detection component to determine a representation of a wakeword; and
in response to determination that the representation of the wakeword, perform speech processing based at least in part on the cancelled data.
14. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine the cancelled data represents speech; and
in response to determination that the cancelled data represents speech, continue use of the first coefficient value for further audio data corresponding to a third time period after the second time period.
15. The system of claim 14 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine the speech has ended; and
in response to determination that the speech has ended, use a second coefficient value for echo cancellation, the second coefficient value corresponding to a fourth time period after the second time period.
16. The system of claim 11 , wherein processing of the third audio data, the fourth audio data, and the first coefficient value to determine the cancelled data corresponds to a first mode of operation and wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine a loudspeaker in an environment of the device is emitting audio; and
in response to the loudspeaker emitting audio, enter the first mode of operation.
17. The system of claim 11 , wherein:
the third audio data includes a first representation of noise detected in an environment of the device;
the fourth audio data includes a second representation of the noise; and
determination of the cancelled data results in at least partial cancellation of the noise as represented in the cancelled data.
18. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
emit audio by a loudspeaker;
receive, from at least one microphone of the device, a fifth audio data including a representation of the audio;
process the fifth audio data to determine acoustic characteristic data corresponding to an environment of the device; and
process the acoustic characteristic data to determine the delay time period.
19. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
receive fifth audio data from a third microphone of the device;
process at least the fourth audio data and the fifth audio data to determine a first matrix represent audio captured by the second microphone and the third microphone; and
determine a second matrix including at least the first coefficient value,
wherein determination of the cancelled data comprises processing the first matrix with respect to the second matrix to determine the cancelled data.
20. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine the cancelled data represents an acoustic event; and
in response to determination that the cancelled data represents the acoustic event, continue use of the first coefficient value for further audio data corresponding to a third time period after the second time period.