Method, apparatus and system for neural network hearing aid
The disclosure generally relates to a method, system and apparatus to improve a user's understanding of speech in real-time conversations by processing the audio through a neural network contained in a hearing device. The hearing device may be a headphone or hearing aid. In one embodiment, the disclosure relates to an apparatus to enhance incoming audio signal. The apparatus includes a controller to receive an incoming signal and provide a controller output signal; a neural network engine (NNE) circuitry in communication with the controller, the NNE circuitry activatable by the controller, the NNE circuitry configured to generate an NNE output signal from the controller output signal; and a digital signal processing (DSP) circuitry to receive one or more of controller output signal or the NNE circuitry output signal to thereby generate a processed signal; wherein the controller determines a processing path of the controller output signal through one of the DSP or the NNE circuitries as a function of one or more of predefined parameters, incoming signal characteristics and NNE circuitry feedback.
1. An ear-worn hearing system residing in an ear-worn housing and configured to enhance an incoming audio signal, comprising:
front-end circuitry configured to receive and digitize the incoming audio signal and output a digitized version of the incoming audio signal;
control circuitry configured to receive the digitized version of the incoming audio signal, estimate a signal-to-noise ratio (SNR) of the digitized version of the incoming audio signal, and output a control circuitry output signal including signal samples;
a digital signal processor (DSP); and
neural network engine (NNE) circuitry switchable into and out of a signal path between the control circuitry and the DSP such that a neural network of the NNE circuitry configured to isolate and de-noise speech in the incoming audio signal is couplable directly to an output of the control circuitry and configured to enhance signal samples in the control circuitry output signal and then output a continuous audio signal based on the enhanced signal samples, wherein the NNE circuitry is further configured to:
determine a target SNR of the continuous audio signal based at least in part on the estimate of the SNR of the digitized version of the incoming audio signal;
apply gains to separated source signals derived from the digitized version of the incoming audio signal, the gains selected based on the target SNR; and
recombine the separated source signals after application of the gains to produce the continuous audio signal.
2. The hearing system of claim 1 , wherein the continuous audio signal is generated in about 32 milliseconds or less of receipt of the incoming audio signal by the hearing system.
3. The hearing system of claim 1 , wherein the continuous audio signal is generated in about 10 milliseconds or less of receipt of the incoming audio signal by the hearing system.
4. The hearing system of claim 1 , wherein the continuous audio signal is generated between about 10-20 ms, 12-8 ms, 10-6 ms or 8-3 milliseconds of receipt of the incoming audio signal by the hearing system.
5. The hearing system of claim 1 , wherein the neural network performs at least 1 billion operations per second.
6. The hearing system of claim 5 , wherein the NNE circuitry is configured to process the digitized version of the incoming audio signal with an associated power consumption of about 2 milliwatts or less.
7. The hearing system of claim 6 , wherein the NNE circuitry is formed on a System-on-Chip (SOC) and further comprises a plurality of non-transitory executable logic.
8. The hearing system of claim 1 , wherein the neural network is configured to enhance the digitized version of the incoming audio signal by estimating a complex ratio mask for each signal sample to obtain a target signal component.
9. The hearing system of claim 8 , wherein the target signal component is speech.
10. The hearing system of claim 9 , wherein the target signal component is one or more recognized speakers.
11. The hearing system of claim 8 , wherein the continuous audio signal exhibits user-configurable decreased background noise compared to the incoming audio signal.
12. The hearing system of claim 10 , further comprising a physical control switch accessible on the hearing system to adjust background noise level of the continuous audio signal.
13. The hearing system of claim 1 , wherein the neural network is configured consume less than 1 milliWatt of power during operation.
14. The hearing system of claim 1 , wherein the neural network of the NNE circuitry configured to isolate and de-noise speech in the incoming audio signal is a first neural network, and wherein the control circuitry comprises a second neural network configured to estimate the SNR.