Contact and acoustic microphones for voice wake and voice processing for AR/VR applications
A method to combine contact and acoustic microphones in a headset for voice wake and voice processing in immersive reality applications is provided. The method includes receiving, from a contact microphone, a first acoustic signal, determining a fidelity and a quality of the first acoustic signal, receiving, from an acoustic microphone, a second acoustic signal, and when the fidelity and quality of the first acoustic signal exceeds a pre-selected threshold, combining the first acoustic signal and the second acoustic signal to provide an enhanced acoustic signal to a smart glass user. A non-transitory, computer-readable medium storing instructions to cause a headset to perform the above method, and the headset, are also provided.
1 . A computer-implemented method, comprising:
receiving, from a first microphone of a client device, a first acoustic signal;
receiving, from a second microphone of the client device, a second acoustic signal;
determining whether the first acoustic signal exceeds a preselected threshold; and
based on determining the first acoustic signal exceeds the preselected threshold, combining the first acoustic signal and the second acoustic signals to generate a third acoustic signal, wherein combining the first acoustic signal and the second acoustic signal comprises permuting the first acoustic signal and the second acoustic signal with weighting factors selected to improve a quality of the third acoustic signal relative to a quality of the first acoustic signal.
2 . The computer-implemented method of claim 1 , wherein:
the first microphone comprises a contact microphone;
the second microphone comprises an acoustic microphone; and
the client device comprises a headset.
3 . The computer-implemented method of claim 1 , further comprising:
wirelessly transmitting the third acoustic signal to the client device; and
receiving, from the client device, a text transcription of a speech of a user derived from the third acoustic signal.
4 . The computer-implemented method of claim 1 , further comprising
determining the first microphone is mechanically coupled to a body part of a user of the client device.
5 . The computer-implemented method of claim 1 , further comprising
identifying a voice activity from a user of the client device with the first acoustic signal when the quality of the first acoustic signal exceeds the preselected threshold.
6 . The computer-implemented method of claim 1 , further comprising
identifying a frequency cutoff based on a noise level of the first microphone below a preselected threshold for frequencies higher than the frequency cutoff, wherein combining the first acoustic signal and the second acoustic signal comprises using the first acoustic signal for frequencies below the frequency cutoff and using the second acoustic signal for frequencies above the frequency cutoff.
7 . The computer-implemented method of claim 1 , wherein
combining the first acoustic signal and the second acoustic signal comprises:
identifying a background interference in the second acoustic signal based on a lack thereof in the first acoustic signal; and
removing the background interference from the third acoustic signal.
8 . The computer-implemented method of claim 1 , wherein
providing the third acoustic signal comprises removing a background interference from the third acoustic signal with a machine learning algorithm fed with the first acoustic signal and the second acoustic signal.
9 . The computer-implemented method of claim 1 , further comprising
identifying a background interference in the second acoustic signal by feeding a waveform pattern including the second acoustic signal through a machine learning algorithm.
10 . The computer-implemented method of claim 1 , further comprising:
receiving a signal from an inertial motion sensor; and
identifying the signal as a background interference in the first acoustic signal.
11 . The computer-implemented method of claim 1 , wherein:
the first acoustic signal and the second acoustic signal are received from a single sound source and at different times of arrival; and
combining the first acoustic signal and the second acoustic signal comprises forming a beam of sound in a direction of the single sound source relative to the client device.
12 . The computer-implemented method of claim 1 , wherein
combining the first acoustic signal and the second acoustic signal comprises selecting a bandwidth of the first acoustic signal and the second acoustic signal to improve a signal-to-noise ratio of the third acoustic signal.
13 . The computer-implemented method of claim 1 , wherein
combining the first acoustic signal and the second acoustic signal comprises permuting the first acoustic signal and the second acoustic signal with weighting factors selected to improve a fidelity of the third acoustic signal relative to a fidelity of the first acoustic signal.
14 . The computer-implemented method of claim 1 , wherein
combining the first acoustic signal and the second acoustic signal comprises spatially filtering the first acoustic signal and the second acoustic signal to form the third acoustic signal in a beam along a selected direction.
15 . A system, comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to perform a method including:
receiving, from a first microphone of a client device, a first acoustic signal;
receiving, from a second microphone of the client device, a second acoustic signal;
determining whether the first acoustic signal exceeds a preselected threshold; and
based on determining the first acoustic signal exceeds the preselected threshold, combining the first acoustic signal and the second acoustic signals to generate a third acoustic signal, wherein combining the first acoustic signal and the second acoustic signal comprises permuting the first acoustic signal and the second acoustic signal with weighting factors selected to improve a quality of the third acoustic signal relative to a quality of the first acoustic signal.
16 . The system of claim 15 , wherein
the method further includes feeding a waveform pattern including the second acoustic signal through a machine learning algorithm to identify a background interference in the second acoustic signal.
17 . The system of claim 15 , wherein
the method further includes:
receiving a signal from an inertial motion sensor; and
identifying the signal as a background interference in the first acoustic signal.
18 . The system of claim 15 , wherein
the method further includes removing a background interference from the third acoustic signal with a machine learning algorithm fed with the first acoustic signal and the second acoustic signal.
19 . The system of claim 15 , wherein
the method further includes:
identifying a background interference in the second acoustic signal based on a lack thereof in the first acoustic signal; and
removing the background interference from the third acoustic signal.
20 . A non-transitory, computer-readable storage medium storing instructions encoded thereon that, when executed by a processor, cause the processor to perform a method comprising:
receiving, from a contact microphone of an augmented reality headset, a first acoustic signal;
receiving, from an acoustic microphone of the augmented reality headset, a second acoustic signal;
determining whether the first acoustic signal exceeds a preselected threshold; and
based on determining the first acoustic signal exceeds the preselected threshold, combining the first acoustic signal and the second acoustic signals to generate a third acoustic signal, wherein combining the first acoustic signal and the second acoustic signal comprises permuting the first acoustic signal and the second acoustic signal with weighting factors selected to improve a fidelity and a quality of the third acoustic signal relative to a fidelity and a quality of the first acoustic signal.