System and method for enhancing speech of target speaker from audio signal in an ear-worn device using voice signatures
An ear-worn device is provided that operates to isolate and individually treat the received speech of a target speaker or multiple target speakers from an audio input signal detected in a multi-speaker environment. The ear-worn device uses a machine learning model that receives a voice signature of each of one or more target speakers as input signals, to identify and isolate the component of the audio input signal attributable to the target speaker(s). Once isolated, the target speaker's speech may be enhanced, de-emphasized, or otherwise processed in a manner desired by the wearer of the ear-worn device. The wearer may use an external electronic device, e.g., a phone, to select one or more target speakers in a conversation and/or configure various settings associated with processing the speech on the ear-worn device.
1. A method of selectively processing, with an ear-worn device including a processor and a microphone coupled to the processor, a target speaker's speech from an audio signal comprising temporally overlapping speech components from multiple speakers, the method comprising:
detecting the audio signal with the microphone of the ear-worn device;
providing the audio signal detected by the microphone of the ear-worn device to the processor of the ear-worn device; and
isolating, with the processor of the ear-worn device, a component of the audio signal representing the target speaker's speech from among the temporally overlapping speech components from multiple speakers by:
providing the audio signal as a first input to a machine learning model and providing a voice signature of the target speaker as a second input to the machine learning model; and
processing the audio signal with the machine learning model using the voice signature of the target speaker.
2. The method of claim 1 , wherein the ear-worn device is a hearing aid or a headphone.
3. The method of claim 1 , wherein the voice signature of the target speaker is a feature vector obtained from another machine learning model trained to discriminate between voices of speakers, and wherein providing the voice signature as the second input to the machine learning model comprises providing the feature vector as an input signal to the machine learning model.
4. The method of claim 1 , wherein the audio signal is a second audio signal, and wherein the method further comprises:
detecting a first audio signal with the microphone of the ear-worn device; and
extracting the voice signature of the target speaker from the first audio signal before processing the second audio signal with the machine learning model using the voice signature of the target speaker.
5. The method of claim 1 , wherein processing the audio signal with the machine learning model comprises processing the audio signal with a neural network.
6. The method of claim 5 , wherein processing the audio signal with a neural network comprises processing the audio signal with a recurrent neural network or a convolutional neural network.
7. The method of claim 1 , wherein the target speaker comprises a wearer of the ear-worn device, and wherein the method further comprises, after isolating the component of the audio signal representing the target speaker's speech from among the temporally overlapping speech components, suppressing the component of the audio signal representing the target speaker's speech.
8. The method of claim 1 , further comprising amplifying the component of the audio signal representing the target speaker's speech after isolating the component of the audio signal representing the target speaker's speech.
9. The method of claim 1 , wherein the temporally overlapping speech components from multiple speakers comprises the component of the audio signal representing the target speaker's speech and a component of the audio signal representing a secondary speaker's speech, and wherein the method further comprises suppressing the component of the audio signal representing the secondary speaker's speech after isolating the component of the audio signal representing the target speaker's speech.
10. The method of claim 1 , wherein the ear-worn device further comprises a speaker device, and wherein the method further comprises outputting from the speaker device of the ear-worn device the component of the audio signal representing the target speaker's speech after isolating the component of the audio signal representing the target speaker's speech from among the temporally overlapping speech components from multiple speakers.
11. An ear-worn device comprising:
one or more microphones configured to detect an audio signal including speech from one or more speakers in a conversation;
at least one processor configured to perform the method of claim 1 to generate an output audio signal; and
one or more speaker devices configured to output the output audio signal.
12. A system, comprising:
the ear-worn device of claim 11 ; and
a second ear-worn device comprising:
one or more second microphones configured to detect a second audio signal;
at least one second processor configured to:
generate a second output audio signal at least in part by:
isolating a component of the second audio signal representing the target speaker's speech from among the temporally overlapping speech components from multiple speakers by processing the second audio signal with a second machine learning model;
receive, from the ear-worn device, the output audio signal; and
combine the output audio signal and the second output audio signal to generate a combined audio signal; and
one or more second speaker devices configured to output the combined audio signal.