Enhanced dynamics processing of streaming audio by source separation and remixing
View Patent ↗This application relates to a systems and methods for enhanced dynamics processing of streaming audio by source separation and remixing for hearing assistance devices, according to one example. In one embodiment, an external streaming audio device processes sources isolated from an audio signal using source separation, and mixes the resulting signals back into the unprocessed audio signal to enhance individual sources while minimizing audible artifacts. Variations of the present system use source separation in a side chain to guide processing of a composite audio signal.
1. A method, comprising:
isolating individual sound source components from an audio signal, including automatically isolating the components based on individual sound source;
independently processing the individual sound source components to compensate for hearing loss of a wearer of a hearing assistance device;
after processing the components, mixing the sound source components with the unprocessed audio signal to produce a mixed audio signal to minimize audible artifacts; and
providing the mixed audio signal to the wearer using the hearing assistance device worn by the wearer.
2. The method of claim 1 , wherein independently processing the individual sound source components includes independently compressing the individual sound source components.
3. The method of claim 1 , wherein independently processing the individual sound source components includes applying source-specific gain processing to the individual sound source components.
4. The method of claim 1 , wherein mixing the compressed sound source components includes providing a mix that is customized to compensate for the wearer's hearing loss.
5. The method of claim 1 , wherein isolating individual sound source components from an audio signal includes using non-negative matrix factorization.
6. The method of claim 1 , further comprising processing the audio signal in parallel with the isolated individual source components before mixing.
7. The method of claim 6 , wherein processing the audio signal includes delaying the audio signal to compensate for latency in source separation.
8. The method of claim 1 , wherein isolating individual sound source components from an audio signal includes processing to isolate voice and instrument components from musical signals.
9. The method of claim 1 , wherein providing the mixed audio signal includes streaming the mixed audio signal to the hearing assistance device.
10. The method of claim 9 , wherein streaming the mixed audio signal to a hearing assistance device includes streaming to a hearing aid.
11. A method, comprising:
isolating individual sound source components from an audio signal, including automatically isolating the components based on individual sound source;
analyzing the isolated sound source components to extract information from the isolated components;
processing the audio signal using the extracted information from the isolated sound sources to guide the processing; and
providing the processed audio signal to a wearer using a hearing assistance device worn by the wearer.
12. The method of claim 11 , wherein processing the audio signal includes using the information from isolated sound source to enhance individual components in the audio signal.
13. The method of claim 11 , wherein using the information from the isolated sound sources includes using source identification.
14. The method of claim 11 , wherein analyzing the isolated sound source components to extract information includes extracting features and characteristics of the sound source components.
15. The method of claim 11 , wherein processing the audio signal using the information includes tuning a bank of resonant filters to enhance individual components in the audio signal.
16. The method of claim 11 , wherein processing the audio signal using the information includes customizing to compensate for the wearer's hearing loss.
17. The method of claim 11 , wherein isolating individual sound source components includes using non-negative matrix factorization.
18. The method of claim 11 , wherein providing the processed audio signal to a wearer using a hearing assistance device includes using a hearing aid.
19. The method of claim 18 , wherein using a hearing aid includes using a behind-the-ear (BTE) hearing aid.
20. The method of claim 18 , wherein using a hearing aid includes using an in-the-ear (ITE) hearing aid.