Multi-channel AEC system identification for self-calibration
Examples described herein relate to calibration of audio playback devices in a media playback system using inputs derived from a multi-channel adaptive filter of an acoustic echo canceller. Example playback devices described herein may utilize one or more techniques for calibration, which may be implemented as various calibration procedures. In some implementations, an example playback device may implement a self-calibration procedure, which involves the playback device calibrating (or re-calibrating) itself during operation. Yet further, the playback device may estimate acoustic impulse responses from a multi-channel adaptive filter of an acoustic echo canceller to use as inputs to the self-calibration procedure.
1. A playback device comprising:
audio transducers;
microphones;
a network interface;
at least one processor; and
a housing carrying the audio transducers, the microphones, the network interface, the at least one processor, and
data storage including instructions that are executable by the at least one processor such that the playback device is configured to:
play back respective audio signals via the audio transducers in a given environment;
during playback of the respective audio signals, capture, via the microphones, respective microphone input streams;
determine, via singular value decomposition, a unitary transformation matrix for the respective audio signals;
determine a reference signal matrix comprising reference signals representing the respective audio signals in a short-time Fourier transform (STFT) domain;
transform, via the determined unitary transformation matrix, the reference signal matrix to at least partially decorrelate the respective audio signals;
determine a measured signal matrix comprising measured signals representing the microphone input streams in the STFT domain;
cancel, via a multi-channel adaptive filter matrix of a multi-channel acoustic echo canceller, at least a portion of the reference signals from the corresponding measured signals;
determine an impulse response matrix as the product of (i) the multi-channel adaptive filter matrix and (ii) a Kronecker product of the unitary transform matrix and an identity matrix;
estimate echo path responses based on the determined impulse response matrix;
determine a calibration that at least partially offsets acoustic characteristics of the given environment as represented by the estimated echo path responses; and
apply the determined calibration to the playback device.
2. The playback device of claim 1 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to estimate the echo path responses based on the determined impulse response matrix comprise instructions the that are executable by the at least one processor such that the playback device is configured to:
determine that first frames of the reference signal matrix have an averaged coherence value that is above a threshold; and
based on the determination, forego update of the estimated echo path responses based on the first frames.
3. The playback device of claim 2 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to estimate the echo path responses based on the determined impulse response matrix comprise instructions the that are executable by the at least one processor such that the playback device is configured to:
determine that second frames of the reference signal matrix have an averaged coherence value that is below the threshold;
determine updates to the impulse response matrix based on states of the multi-channel adaptive filter matrix corresponding to the second frames;
update the estimated echo path responses based on the updates to the impulse response matrix; based on the determination; and
as the estimated echo path responses are updated, apply a smoothing function to the updates.
4. The playback device of claim 2 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to determine that the particular frames of the reference signal matrix have the averaged coherence value is above the threshold comprise instructions the that are executable by the at least one processor such that the playback device is configured to:
determine respective coherence values between the n th frames of the reference signals in the reference signal matrix; and
average the respective coherence values across a particular frequency range.
5. The playback device of claim 4 , wherein the particular frequency range is approximately 50-500 Hz.
6. The playback device of claim 1 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to estimate echo path responses based on the determined impulse response matrix further comprise instructions the that are executable by the at least one processor such that the playback device is configured to:
provide a delta to the impulse response matrix to generate signals representing the estimated echo path responses;
condition the signals representing the estimated echo path responses to conform to an input specification of a self-calibrator, wherein the instructions that are executable by the at least one processor such that the playback device is configured to determine the calibration further comprise instructions the that are executable by the at least one processor such that the playback device is configured
provide the conditioned signals estimated echo path responses to the self-calibrator; and
determine, via the self-calibrator, the calibration.
7. The playback device of claim 6 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to condition the estimated echo path responses to conform to the input specification of the self-calibrator further comprise instructions the that are executable by the at least one processor such that the playback device is configured to:
apply octave smoothing to the estimated echo path responses.
8. The playback device of claim 6 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to determine the calibration further comprise instructions the that are executable by the at least one processor such that the playback device is configured to:
query a dataset for particular stored acoustic responses that correspond to the estimated echo path responses, wherein the dataset relates a plurality of stored acoustic responses to respective calibrations.
9. The playback device of claim 8 , wherein the data storage comprises the dataset, and wherein the plurality of stored acoustic responses are determined based on multiple media playback systems each performing a respective acoustic room response determination process comprising (i) outputting, via a respective playback device within a respective environment that is not the same as the environment in which the playback device is located, respective audio content, (ii) while the respective playback device outputs the respective audio content, captures, via a first microphone disposed in a housing of the respective playback device, respective first audio data representing reflections of the respective audio content in the respective environment, (iii) captures, via a second microphone disposed in a housing of a respective mobile device, respective second audio data representing reflections of the respective audio content in the respective environment, (iv) based on the respective first audio data, determines an acoustic response of the respective environment, and (v) based on the respective second audio data, determines a calibration of the respective playback device in the respective environment.
10. The playback device of claim 8 , wherein a database comprises the dataset, and wherein the instructions that are executable by the at least one processor such that the playback device is configured to query the dataset for the particular stored acoustic response comprise instructions that are executable by the at least one processor such that the playback device is configured to:
send, via the network interface to a server, a query of the database.
11. The playback device of claim 8 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to query the dataset for the particular stored acoustic response comprise instructions that are executable by the at least one processor such that the playback device is configured to:
map the estimated echo path responses to the particular stored acoustic responses in the dataset that satisfy a threshold similarity to the estimated echo path responses.
12. The playback device of claim 1 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to cancel at least the portion of the reference signals from the corresponding measured signals further comprise instructions the that are executable by the at least one processor such that the playback device is configured to:
before estimation of the n th frame of the multi-channel adaptive filter matrix, apply an error recovery non-linearity function to the error signal matrix to estimate true error signals from the error signal matrix, wherein the n th frame of the multi-channel adaptive filter matrix is based on the estimated true error signals.
13. The playback device of claim 1 , wherein the playback device further comprises a digital-to-analog converter, and wherein the playback device further comprises instructions that are executable by the at least one processor such that the playback device is configured to:
receive, via the network interface, data representing audio content; and
convert, via the digital-to-analog converter, the data to the respective audio signals for the audio transducers.
14. The playback device of claim 1 , wherein the multi-channel acoustic echo canceller outputs STFT-domain output signals, and wherein data storage further comprises instructions that are executable by the at least one processor such that the playback device is configured to:
convert the STFT-domain output signals to time-domain output signals; and
cause a voice assistant to process the time-domain output signals.
15. The playback device of claim 1 , wherein the instructions that are executable by the at least one processor such that the playback device is configured to cancel at least the portion of the reference signals from the corresponding measured signals comprise instructions that are executable by the at least one processor such that the playback device is configured to:
during each i th iteration of the multi-channel acoustic echo canceller:
(1) determine n th frames of a model signal matrix by applying an n−1 th frame of a multi-channel adaptive filter matrix to the n th frames of the reference signal matrix;
(2) determine n th frames of an error signal matrix representing respective error between the model signal matrix and the measured signal matrix; and
(3) determine an n th frame of the multi-channel adaptive filter matrix as a sum of (a) the n−1 th frame of the multi-channel adaptive filter matrix and (b) a dot product of (i) an adaptive step-size matrix and (ii) a product of the error signal matrix and the transformed reference signal matrix.
16. A system comprising:
a playback device comprising audio transducers and microphones;
a network interface;
at least one processor; and
data storage including instructions that are executable by the at least one processor such that the system is configured to:
play back respective audio signals via the audio transducers in a given environment;
during playback of the respective audio signals, capture, via the microphones, respective microphone input streams;
determine, via singular value decomposition, a unitary transformation matrix for the respective audio signals;
determine a reference signal matrix comprising reference signals representing the respective audio signals in a short-time Fourier transform (STFT) domain;
transform, via the determined unitary transformation matrix, the reference signal matrix to at least partially decorrelate the respective audio signals;
determine a measured signal matrix comprising measured signals representing the microphone input streams in the STFT domain;
cancel, via a multi-channel adaptive filter matrix of a multi-channel acoustic echo canceller, at least a portion of the reference signals from the corresponding measured signals;
determine an impulse response matrix as the product of (i) the multi-channel adaptive filter matrix and (ii) a Kronecker product of the unitary transform matrix and an identity matrix;
estimate echo path responses based on the determined impulse response matrix;
determine a calibration that at least partially offsets acoustic characteristics of the given environment as represented by the estimated echo path responses; and
apply the determined calibration to the playback device.
17. The system of claim 16 , wherein the instructions that are executable by the at least one processor such that the system is configured to estimate the echo path responses based on the determined impulse response matrix comprise instructions the that are executable by the at least one processor such that the system is configured to:
determine that first frames of the reference signal matrix have an averaged coherence value that is above a threshold; and
based on the determination, forego update of the estimated echo path responses based on the first frames.
18. The system of claim 17 , wherein the instructions that are executable by the at least one processor such that the system is configured to estimate the echo path responses based on the determined impulse response matrix comprise instructions the that are executable by the at least one processor such that the system is configured to:
determine that second frames of the reference signal matrix have an averaged coherence value that is below the threshold;
determine updates to the impulse response matrix based on states of the multi-channel adaptive filter matrix corresponding to the second frames;
update the estimated echo path responses based on the updates to the impulse response matrix; based on the determination; and
as the estimated echo path responses are updated, apply a smoothing function to the updates.
19. The system of claim 17 , wherein the instructions that are executable by the at least one processor such that the system is configured to determine that the particular frames of the reference signal matrix have the averaged coherence value is above the threshold comprise instructions the that are executable by the at least one processor such that the system is configured to:
determine respective coherence values between the n th frames of the reference signals in the reference signal matrix; and
average the respective coherence values across a particular frequency range.
20. A method comprising:
playing back respective audio signals via audio transducers in a given environment;
during playback of the respective audio signals, capturing, via microphones, respective microphone input streams;
determining, via singular value decomposition, a unitary transformation matrix for the respective audio signals;
determining a reference signal matrix comprising reference signals representing the respective audio signals in a short-time Fourier transform (STFT) domain;
transforming, via the determined unitary transformation matrix, the reference signal matrix to at least partially decorrelate the respective audio signals;
determining a measured signal matrix comprising measured signals representing the microphone input streams in the STFT domain;
cancelling, via a multi-channel adaptive filter matrix of a multi-channel acoustic echo canceller, at least a portion of the reference signals from the corresponding measured signals;
determining an impulse response matrix as the product of (i) the multi-channel adaptive filter matrix and (ii) a Kronecker product of the unitary transform matrix and an identity matrix;
estimating echo path responses based on the determined impulse response matrix;
determining a calibration that at least partially offsets acoustic characteristics of the given environment as represented by the estimated echo path responses; and
applying the determined calibration to a playback device.