Ear-worn device with neural network-based noise modification and/or spatial focusing
An ear-worn device includes two or more microphones and noise reduction circuitry including neural network circuitry. The neural network circuitry is configured to: receive multiple audio signals wherein at least two of the multiple audio signals each originate from a different one of the two or more microphones and/or at least one of the multiple audio signals is a beamformed audio signal originating from the two or more microphones; and implement one or more neural network layers trained to perform background noise modification and spatial focusing based on the multiple audio signals, such that the neural network circuitry generates, based on the multiple audio signals, one or more neural network outputs. The noise reduction circuitry is configured to output, based on the one or more neural network outputs, an output audio signal comprising a background noise-modified and spatially-focused version of a first audio signal of the multiple audio signals.
1 . An ear-worn device, comprising:
two or more microphones; and
noise reduction circuitry comprising neural network circuitry,
wherein the neural network circuitry is configured to:
receive multiple audio signals wherein at least two of the multiple audio signals each originate from a different one of the two or more microphones and/or at least one of the multiple audio signals is a beamformed audio signal originating from the two or more microphones; and
implement one or more neural network layers trained to perform background noise modification and spatial focusing, such that the neural network circuitry generates, based on the multiple audio signals, two or more neural network outputs,
wherein the noise reduction circuitry is configured to generate an output audio signal comprising:
a target speech signal comprising a first spatially-focused version of a speech signal, wherein the speech signal comprises speech in a first audio signal among the multiple audio signals;
an interfering speech signal comprising a second spatially-focused version of the speech signal; and
a background noise signal comprising background noise in the first audio signal,
wherein the noise reduction circuitry is further configured to generate the output audio signal such that, in the output audio signal:
a change in volume of the background noise signal is different from a change in volume of the target speech signal by a first volume change difference amount;
a change in volume of the interfering speech signal is different from the change in volume of the target speech signal by a second volume change difference amount; and
the first volume change difference amount and the second volume change difference amount are independently controllable,
wherein the neural network circuitry is further configured to use:
a first subset of the one or more neural network layers to generate a first of the two or more neural network outputs; and
a second subset of the one or more neural network layers to generate a second of the two or more neural network outputs, and
wherein the noise reduction circuitry is further configured to obtain the speech signal and/or the background noise signal from the first of the two or more neural network outputs, and to obtain the target speech signal and/or the interfering speech signal from the second of the two or more neural network outputs.
2 . The ear-worn device of claim 1 , wherein at least two of the multiple audio signals have different beamformed directional patterns.
3 . The ear-worn device of claim 1 , wherein the target speech signal comprises the speech signal to which has been applied a particular spatial focusing pattern, the particular spatial focusing pattern comprising different weights applied to the speech originating from different directions-of-arrival relative to a wearer of the ear-worn device.
4 . The ear-worn device of claim 3 , wherein the particular spatial focusing pattern comprises higher weights applied to speech originating from directions-of-arrival towards a front of the wearer of the car-worn device than weights applied to speech originating from directions-of-arrival towards sides and a back of the wearer.
5 . The ear-worn device of claim 3 , wherein the neural network circuitry is further configured to:
receive one or more spatial focusing control inputs indicating the particular spatial focusing pattern; and
use the one or more spatial focusing control inputs to generate the two or more neural network outputs such that the target speech signal comprises the speech signal to which has been applied the particular spatial focusing pattern.
6 . The ear-worn device of claim 5 , further comprising:
communication circuitry configured to receive, from a processing device, an indication of a user selection of the particular spatial focusing pattern; and
control circuitry configured to generate, based at least in part on the indication of the user selection of the particular spatial focusing pattern, the one or more spatial focusing control inputs indicating the particular spatial focusing pattern.
7 . A system comprising:
the ear-worn device of claim 6 ; and
the processing device in communication with the ear-worn device and configured to:
display a graphical user interface including options for different spatial focusing patterns; and
receive the user selection of the particular spatial focusing pattern.
8 . The ear-worn device of claim 1 , wherein the interfering speech signal comprises a remainder when the target speech signal is subtracted from the speech signal.
9 . The ear-worn device of claim 1 , wherein the two or more neural network outputs comprise two different masks.
10 . The ear-worn device of claim 1 , further comprising:
mixing circuitry configured to:
generate the output audio signal by mixing a combination of audio signals; or
generate the output audio signal by mixing a combination of masks; or
wide dynamic range compression (WDRC) circuitry comprising multiple WDRC pipelines configured to generate the output audio signal by performing WDRC on the combination of audio signals.
11 . The ear-worn device of claim 10 , wherein the mixing circuitry is further configured to:
receive a first volume change control input and a second volume change control input; and
perform the mixing using the first volume change control input and the second volume change control input such that the first volume change difference amount is controlled, at least in part, by the first volume change control input and the second volume change difference amount is controlled, at least in part, by the second volume change control input.
12 . The ear-worn device of claim 11 , further comprising:
communication circuitry configured to receive the first volume change control input and the second volume change control input from a processing device;
memory configured to store the first volume change control input and the second volume change control input; and
control circuitry configured to retrieve the first volume change control input and the second volume change control input from the memory and output the first volume change control input and the second volume change control input to the mixing circuitry.
13 . The ear-worn device of claim 11 , further comprising:
control circuitry configured to:
generate the first volume change control input based on a level of background noise in the first audio signal; and
generate the second volume change control input based on a level of interfering speech in the first audio signal.
14 . The ear-worn device of claim 1 , wherein the ear-worn device is further configured to receive a user selection to turn spatial focusing off.
15 . The ear-worn device of claim 1 ,
wherein at least one of the two or more neural network outputs comprises:
the speech signal;
a mask configured to generate the speech signal;
the background noise signal;
a mask configured to generate the background noise signal;
the target speech signal;
a mask configured to generate the target speech signal;
the interfering speech signal; or
a mask configured to generate the interfering speech signal.
16 . The ear-worn device of claim 1 , wherein:
the background noise signal is not spatially-focused; and
the interfering speech signal does not comprise a portion of the background noise in the first audio signal.
17 . The ear-worn device of claim 1 , wherein:
the background noise signal comprises a first spatially-focused version of the background noise in the first audio signal; and
the interfering speech signal comprises the second spatially-focused version of the speech signal plus a second spatially-focused version of the background noise in the first audio signal.
18 . The ear-worn device of claim 1 , wherein the ear-worn device comprises a hearing aid.
19 . The ear-worn device of claim 1 , wherein the noise reduction circuitry is implemented on a chip.