Perceptually guided speech enhancement using deep neural networks

1. A system, comprising:

an electronic device configured to receive at least one sound;

at least one sensor communicatively coupled to the electronic device, the at least one sensor configured to sense an input from at least one user of the electronic device in response to the received at least one sound; and

a processor communicatively coupled to the electronic device configured to enhance the at least one sound for the at least one user based on a compound metric using a neural network, the compound metric calculated using a physiological metric and at least one sound metric selected from an engineering metric and a perceptual metric, the engineering metric comprising a difference between an output signal and a desired signal.

2. The system of claim 1 , wherein the processor is further configured to train the neural network using training data to build a compound metric model.

3. The system of claim 2 , wherein the training data is based on a at least one user characteristic.

4. The system of claim 3 , wherein the at least one user characteristic comprises a user age range.

5. The system of claim 1 , wherein the processor is further configured to train the neural network based on the user input.

6. The system of claim 1 , wherein the user input comprises one or more of an audible input, a gesture input, and a tactile input.

7. The system of claim 1 , wherein the perceptual metric is calculated by using at least one of a short term objective intelligibility metric (STOI), a hearing-aid speech quality index (HASQI), a hearing-aid speech perception index (HASPI), a perceptual evaluation of speech quality (PESQ), and a perceptual evaluation of audio quality (PEAQ).

8. The system of claim 1 , wherein the compound metric is calculated during a predetermined testing time period.

9. The system of claim 8 , wherein the compound metric is calculated at a plurality of time periods subsequent to the predetermined testing time period.

10. The system of claim 1 , wherein the electronic device is an ear-worn electronic device configured to be worn by the at least one user.

11. The system of claim 1 , wherein the electronic device is a mobile communication device.

12. The system of claim 1 , wherein the at least one sound comprises one or more of speech, music, and an alarm.

13. The system of claim 1 , further comprising calculating the compound metric in response to the user input.

14. The system of claim 1 , further comprising calculating the compound metric in response to the difference between the output signal and the clean signal being greater than a predetermined threshold.

15. The system of claim 1 , wherein the physiological metric comprises at least one of an electroencephalogram (EEG) signal, a skin conductance signal, and a heart rate signal.

16. A method, comprising:

receiving at least one sound at an electronic device; and

using a neural network, enhancing the at least one sound for at least one user based on a compound metric, the compound metric calculated using a physiological metric and at least one of an engineering metric and a perceptual metric, the perceptual metric and the physiological metric based at least in part on input sensed from the at least one user in response to the received at least one sound.

17. The method of claim 16 , wherein enhancing the at least one sound for the at least one user comprises training the neural network using training data to build a compound metric model.

18. The method of claim 17 , wherein the training data is based on a at least one user characteristic.

19. The method of claim 16 , wherein enhancing the at least one sound for the at least one user comprises training the neural network based on the user input.

20. The method of claim 16 , wherein the user input comprises one or more of an audible input, a gesture input, and a tactile input.