Audio processing using an intelligent microphone
The present disclosure relates generally to improving audio processing using an intelligent microphone and, more particularly, to techniques for processing audio received at a microphone with integrated analog-to-digital conversion, digital signal processing, acoustic source separation, and for further processing by a speech recognition system. Embodiments of the present disclosure include intelligent microphone systems designed to collect and process high-quality audio input efficiently. Systems and method for audio processing using an intelligent microphone include an integrated package with one or more microphones, analog-to-digital converters (ADCs), digital signal processors (DSPs), source separation modules, memory, and automatic speech recognition. Systems and methods are also provided for audio processing using an intelligent microphone that includes a microphone array and uses a preprogrammed audio beamformer calibrated to the included microphone array.
1. An integrated circuit for providing advanced speech processing as a single package, comprising:
a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array;
circuitry for processing the respective audio signals, the circuitry comprising:
source separation module for separating sources present in the respective audio signals and generating one or more improved audio signals;
automatic speech recognition module for extracting speech information in the one or more improved audio signals and determining confidence level of extracted speech information; and
a feedback path for the confidence level to control one or more parameters affecting the source separation module.
2. The integrated circuit of claim 1 , wherein:
the source separation module determines a level of noise in the respective audio signals; and
the circuitry further includes a feedforward path for the level of the noise to control the automatic speech recognition module.
3. The integrated circuit of claim 1 , further comprising:
one or more sensors for sensing a source near the microphone array;
wherein the circuitry further includes a control path for an output of the one or more sensors to control the automatic speech recognition module.
4. The integrated circuit of claim 1 , wherein:
the circuitry for processing the respective audio signals further comprises a voice activity detecting module for detecting voice activity in the respective audio signals; and
the circuitry further includes a control path for an output of the voice activity detecting module to control the automatic speech recognition module.
5. An integrated circuit for providing advanced speech processing as a single package, comprising:
a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and
circuitry for processing the respective audio signals, the circuitry comprising:
a noise reduction module for reducing noise in the respective audio signals and generating one or more improved audio signals;
an automatic speech recognition module for extracting speech information in the one or more improved audio signals and determining confidence level of extracted speech information; and
a feedback path for the confidence level to control one or more parameters affecting sensitivity of the noise reduction module.
6. The integrated circuit of claim 5 , wherein:
the circuitry for processing the respective audio signals further comprises:
a source separation module for separating sources present in the respective audio signals and determining a level of noise in the one or more improved audio signals; and
a feedback path for the level of noise to control one or more parameters affecting sensitivity of the noise reduction module.
7. The integrated circuit of claim 5 , wherein:
the noise reduction module determines a level of noise in the respective audio signals; and
the circuitry further includes a feedforward path for the level of the noise to control the automatic speech recognition module.
8. An integrated circuit for providing advanced speech processing as a single package, comprising:
a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and
circuitry for processing the respective audio signals comprises:
a beamforming module for adaptively steering a beam of the microphone array;
a source separation module for separating sources present in the respective audio signals and determining a level of noise in the respective audio signals; and
a feedback path for the level of noise to control one or more parameters affecting the beam being formed by the beamforming module.
9. An integrated circuit for providing advanced speech processing as a single package, comprising:
a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and
circuitry for processing the respective audio signals comprises:
a beamforming module for adaptively steering a beam of the microphone array;
a noise reduction module for determining a level of noise in the respective audio signals; and
a feedback path for the level of noise to control one or more parameters affecting the beam being formed by the beamforming module.
10. An integrated circuit for providing advanced speech processing as a single package, comprising:
a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and
one or more sensors for sensing a source near the microphone array;
circuitry for processing the respective audio signals, the circuitry comprising:
a beamforming module for adaptively steering a beam of the microphone array;
an automatic speech recognition module for extracting speech information in the respective audio signals and determining confidence level of extracted speech information;
a feedback path for the confidence level to control one or more parameters affecting the beam being formed by the beamforming module; and
a control path for an output of the one or more sensors to control one or more parameters affecting the beam being formed by the beamforming module.
11. The integrated circuit of claim 10 , wherein the microphone array is a quad microphone comprising four microphone elements arranged in a geometric configuration.
12. An integrated circuit for providing advanced speech processing as a single package, comprising:
a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and
circuitry for processing the respective audio signals, the circuitry comprising:
a beamforming module for adaptively steering a beam of the microphone array;
an automatic speech recognition module for extracting speech information in the respective audio signals and determining confidence level of extracted speech information;
a feedback path for the confidence level to control one or more parameters affecting the beam being formed by the beamforming module;
a voice activity detecting module for detecting voice activity in the respective audio signals; and
a control path for an output of the voice activity detecting module to control one or more parameters affecting the beam being formed by the beamforming module.
13. An integrated circuit for providing advanced speech processing as a single package, comprising:
a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array;
circuitry for processing the respective audio signals, the circuitry comprising:
a beamforming module for adaptively steering a beam of the microphone array;
an automatic speech recognition module for extracting speech information in the respective audio signals and determining confidence level of extracted speech information;
a feedback path for the confidence level to control one or more parameters affecting the beam being formed by the beamforming module; and
a motion sensing module for sensing movement of an electronic device having the microphone array and the circuitry; and
wherein the beamforming module determines changes in position or orientation of the microphone array relative to an audio source of interest based on the respective audio signals and movement information from the motion sensing module adjusts beamforming functionality in response to the changes.
14. A method for audio processing within a single integrated package, the method comprising:
generating audio signals by microphones of a microphone array within the single integrated package; and
processing the audio signals by circuitry in the same integrated package as the microphone array, wherein:
the circuitry in the same integrated package includes two or more of the following modules: source separation module, automatic speech recognition module, adaptive beamforming module, noise reduction module, and voice activity detection module; and
the processing includes:
determining a confidence level of extracted speech information by the automatic speech recognition module; and
controlling one or more parameters affecting a sensitivity of the source separation module based on the confidence level.
15. The method of claim 14 , further comprising:
transmitting over a network interface the processed audio signals to a remote automatic speech recognition service over a network.
16. The method of claim 14 , wherein the processing comprises:
controlling, based on the confidence level, one or more parameters affecting one or more of: a sensitivity of the noise reduction module and a beam formed by the adaptive beamforming module.
17. A method for audio processing within a single integrated package, the method comprising:
generating audio signals by microphones of a microphone array within the single integrated package; and
processing the audio signals by circuitry in the same integrated package as the microphone array, wherein:
the circuitry in the same integrated package includes two or more of the following modules: source separation module, automatic speech recognition module, adaptive beamforming module, noise reduction module, and voice activity detection module; and
the processing includes:
determining a level of noise by one or more of: the source separation module and the noise reduction module; and
controlling one or more parameters affecting a beam formed by the adaptive beamforming module based on the level of noise.
18. A method for audio processing within a single integrated package, the method comprising:
generating audio signals by microphones of a microphone array within the single integrated package; and
processing the audio signals by circuitry in the same integrated package as the microphone array, wherein:
the circuitry in the same integrated package includes two or more of the following modules: source separation module, automatic speech recognition module, adaptive beamforming module, noise reduction module, and voice activity detection module; and
the processing includes:
determining a level of noise by one or more of: the source separation module and the noise reduction module; and
controlling the automatic speech recognition module based on the level of noise.
19. The method of claim 18 , wherein controlling the automatic speech recognition module to stop extracting speech information if the level of noise is greater than a threshold.
20. The method of claim 18 , wherein controlling the automatic speech recognition module comprises requesting the automatic speech recognition module to only output speech information extracted above a certain confidence level if the level of noise is greater than a threshold.