Microphone array auto-directive adaptive wideband beamforming using orientation information from MEMS sensors
A method and apparatus for auto-directive adaptive beamforming for a microphone array using microelectromechanical systems (MEMS) sensor orientation information are provided. The microphone array captures audio and the MEMS sensor detects an orientation of the microphone array. A direction of arrival of a source signal is estimated based on the data representative of the audio. A change in an orientation of the microphone array is detected based on the orientation and the direction of arrival is compensates based on the change in the orientation of the microphone array. The apparatus pre-steers a beam of a beam pattern of the microphone array based on the compensated direction of arrival to retain the source signal in a broadside of the microphone array and performs adaptive wideband beamforming to null one or more interfering sources in the beam pattern while retaining the source signal in the broadside of the microphone array.
1. A device, comprising:
a microphone array including a plurality of microphones configured to capture audio and output one or more signals representative of the audio;
a microelectromechanical systems (MEMS) sensor configured to detect an orientation of the microphone array and output data representative of the orientation;
an audio decoder configured to receive the one or more signals, process the one or more signals and output data representative of the audio; and
a processor configured to:
receive the data representative of the audio and the data representative of the orientation, the data representative of the audio including a plurality of frames;
estimate a direction of arrival of a source signal based on the data representative of the audio by:
performing a short-time Fourier transform (STFT) on a frame of the plurality of frames to produce a transformed frame;
determining whether the transformed frame has a signal power above a noise threshold; and
in response to the signal power being above the noise threshold, estimating the direction of arrival of the source signal;
detect, based on the data representative of the orientation, a change in the orientation of the microphone array;
compensate the direction of arrival based on the change in the orientation of the microphone array;
pre-steer a beam of a beam pattern of the microphone array based on the compensated direction of arrival to retain the source signal in a broadside of the microphone array; and
perform adaptive wideband beamforming to null one or more interfering sources in the beam pattern while retaining the source signal in the broadside of the microphone array.
2. The device according to claim 1 , wherein the processor is configured to:
in response to the signal power being above the noise threshold, determine whether the frame has voice activity; and
in response to the frame having the voice activity, estimate the direction of arrival of the source signal.
3. The device according to claim 2 , wherein the processor is configured to:
before determining whether the frame has the voice activity, perform a coherence test to determine whether the frame is dominated by a single source; and
on a condition that the frame is dominated by the single source, determine whether the frame has the voice activity.
4. The device according to claim 1 , wherein the MEMS sensor is an accelerometer, gyroscope, magnetometer, inertial sensor or magnetic compass.
5. The device according to claim 1 , wherein the processor is configured to generate a histogram of a plurality of directions of arrival of the plurality of frames.
6. The device according to claim 5 , wherein the processor is configured to determine a dominant source direction of arrival based on the histogram.
7. The device according to claim 6 , wherein the processor is configured to determine the dominant source direction of arrival as a mean, median or mode of the histogram.
8. A system, comprising:
a processor; and
non-transitory computer-readable storage media having stored thereon executable instructions that, when executed by the processor cause the processor to:
receive data representative of audio captured by a microphone array including a plurality of microphones, the data representative of the audio including a plurality of frames;
receive data representative of an orientation of the microphone array detected by a microelectromechanical systems (MEMS) sensor;
estimate a direction of arrival of a source signal based on the data representative of the audio by:
performing a short-time Fourier transform (STFT) on a frame of the plurality of frames to produce a transformed frame;
determining whether the transformed frame has a signal power above a noise threshold; and
in response to the signal power being above the noise threshold, estimating the direction of arrival of the source signal;
detect, based on the data representative of the orientation, a change in the orientation of the microphone array;
compensate the direction of arrival based on the change in the orientation of the microphone array;
pre-steer a beam of a beam pattern of the microphone array based on the compensated direction of arrival to retain the source signal in a broadside of the microphone array; and
perform adaptive wideband beamforming to null one or more interfering sources in the beam pattern while retaining the source signal in the broadside of the microphone array.
9. The system according to claim 8 , wherein the instructions cause the processor to estimate the direction of arrival of the source signal by:
in response the signal power being above the noise threshold, determining whether the frame has voice activity; and
in response to the frame having the voice activity, estimating the direction of arrival of the source signal.
10. The system according to claim 9 , wherein the instructions cause the processor to:
before determining whether the frame has the voice activity, perform a coherence test to determine whether the frame is dominated by a single source; and
on a condition that the frame is dominated by the single source, determine whether the frame has the voice activity.
11. The system according to claim 8 , wherein the MEMS sensor is an accelerometer, gyroscope, magnetometer, inertial sensor or magnetic compass.
12. The system according to claim 8 , wherein the instructions cause the processor to generate a histogram of a plurality of directions of arrival of the plurality of frames.
13. The system according to claim 12 , wherein the instructions cause the processor to determine a dominant source direction of arrival based on the histogram.
14. The system according to claim 13 , wherein the instructions cause the processor to determine the dominant source direction of arrival as a mean, median or mode of the histogram.
15. A method, comprising:
receiving data representative of audio captured by a microphone array including a plurality of microphones, the data representative of the audio including a plurality of frames;
receiving data representative of an orientation of the microphone array detected by a microelectromechanical systems (MEMS) sensor;
estimating a direction of arrival of a source signal based on the data representative of the audio by:
performing a short-time Fourier transform (STFT) on a frame of the plurality of frames to produce a transformed frame;
determining whether the transformed frame has a signal power above a noise threshold; and
in response to the signal power being above the noise threshold, estimating the direction of arrival of the source signal;
detecting, based on the data representative of the orientation, a change in the orientation of the microphone array;
compensating the direction of arrival based on the change in the orientation of the microphone array;
pre-steering a beam of a beam pattern of the microphone array based on the compensated direction of arrival to retain the source signal in a broadside of the microphone array; and
performing adaptive wideband beamforming to null one or more interfering sources in the beam pattern while retaining the source signal in the broadside of the microphone array.
16. The method according to claim 15 , wherein estimating the direction of arrival of the source signal includes:
in response to the signal power being above the noise threshold, determining whether the frame has voice activity; and
in response to the frame having the voice activity, estimating the direction of arrival of the source signal.
17. The method according to claim 16 , comprising:
before determining whether the frame has the voice activity, performing a coherence test to determine whether the frame is dominated by a single source; and
on a condition that the frame is dominated by the single source, determining whether the frame has the voice activity.