Method, apparatus and system for neural network hearing aid
The disclosure generally relates to a method, system and apparatus to improve a user's understanding of speech in real-time conversations by processing the audio through a neural network contained in a hearing device. The hearing device may be a headphone or hearing aid. In one embodiment, the disclosure relates to an apparatus to enhance incoming audio signal. The apparatus includes a controller to receive an incoming signal and provide a controller output signal; a neural network engine (NNE) circuitry in communication with the controller, the NNE circuitry activatable by the controller, the NNE circuitry configured to generate an NNE output signal from the controller output signal; and a digital signal processing (DSP) circuitry to receive one or more of controller output signal or the NNE circuitry output signal to thereby generate a processed signal; wherein the controller determines a processing path of the controller output signal through one of the DSP or the NNE circuitries as a function of one or more of predefined parameters, incoming signal characteristics and NNE circuitry feedback.
1. An ear-worn hearing system to enhance quality of an incoming audio signal, comprising:
user-configurable neural network engine (NNE) circuitry configured to receive the incoming audio signal and provide a continuous output signal based on processing of the incoming audio signal, the continuous output signal having a user-adjustable background noise level; and
a user-adjustable control configured to receive input from a user of the ear-worn hearing system to adjust the user-adjustable background noise level,
wherein the ear-worn hearing system is configured to manipulate a configuration of the NNE circuitry in response to the user-adjustable control receiving input from the user.
2. The ear-worn hearing system of claim 1 , wherein the user-adjustable control comprises an adjustable physical dial.
3. The ear-worn hearing system of claim 1 , wherein the user-adjustable control is configured to adjust a signal-to-noise ratio (SNR) of the continuous output signal.
4. The ear-worn hearing system of claim 1 , wherein the user-adjustable control is configured to exclusively affect a noise component of the incoming audio signal or exclusive affect a speech component of the incoming audio signal.
5. An apparatus to enhance quality of an audio signal, the apparatus comprising:
neural network engine (NNE) circuitry comprising a source separation module configured to receive an input audio signal and output one or more intermediate signals representing one or more portions of the input audio signal corresponding to one or more respective sound sources;
a relative gain module configured to receive the one or more intermediate signals and apply a respective gain or gains to the one or more intermediate signals; and
a recombiner module configured to receive the one or more intermediate signals from the relative gain module after application of the respective gain or gains by the relative gain module, the recombiner module further configured to combine the one or more intermediate signals into a combined signal,
wherein the respective gain or gains are set based on a target signal-to-noise ratio (SNR) of the combined signal and wherein the SNR is determined as a function of at least one user-specific criteria and at least one user-agnostic criteria.
6. The apparatus of claim 5 , wherein the user-specific criteria comprises volume targets for signal and noise classes.
7. The apparatus of claim 6 , wherein the volume targets are user controlled.
8. The apparatus of claim 5 , wherein a number and composition of the one or more intermediate signals as output by the source separation module are user-configurable.
9. The apparatus of claim 5 , wherein the user-specific criteria comprises a target amplification of one or more natural speakers.
10. The apparatus of claim 5 , wherein the user-agnostic criteria comprises an estimated SNR of a previously received and processed input audio signal.
11. The apparatus of claim 6 , wherein the user-agnostic criteria comprises an estimated error of the neural network engine circuitry.
12. The apparatus of claim 11 , wherein the recombiner module forms the combined signal based on predicted error of the neural network engine circuitry.
13. The apparatus of claim 5 , wherein the target SNR is determined as a lower of a user's desired SNR or an SNR based on an estimated error of the neural network engine circuitry.
14. The apparatus of claim 5 , wherein the user-specific criteria comprises a target ratio of volumes between sound classes and SNR.
15. The apparatus of claim 5 , wherein NNE circuitry comprises the recombiner module.
16. The apparatus of claim 15 , wherein the relative gain module and/or the recombiner module are user-configurable.
17. The apparatus of claim 5 , wherein the one or more intermediate signals comprises one or more speech components of the input audio signal, wherein the source separation module is further configured to subtract the one or more speech components from the input audio signal to obtain a noise component, and wherein the relative gain module is configured to increase a relative gain of the one or more speech components compared to the noise component.
18. The ear-worn hearing system of claim 1 , wherein the ear-worn hearing system comprises a mobile device, and wherein the user-adjustable control is a graphical user interface (GUI) element displayed on the mobile device.
19. The ear-worn hearing system of claim 18 , wherein the NNE circuitry is contained by the mobile device.
20. The ear-worn hearing system of claim 18 , wherein the ear-worn hearing system comprises an earpiece and wherein the NNE circuitry is contained in the earpiece.