Audio noise removal using one or more neural networks
Apparatuses, systems, and techniques are presented to reduce noise in audio. In at least one embodiment, a sequence of neural networks is used to remove foreground and background noise from audio including a primary audio signal.
1. One or more processors, comprising:
one or more circuits to:
cause one or more digital signals to be filtered, where one or more portions of speech are removed from the one or more filtered digital signals;
generate, by one or more neural networks, the one or more portions of speech filtered from the one or more digital signals based, at least in part, on the one or more filtered digital signals; and
add the one or more portions of speech, generated by the one or more neural networks, to the one or more filtered digital signals.
2. The one or more processors of claim 1 , wherein the one or more circuits are to cause one or more second neural networks to filter noise from the one or more digital signals, the one or more second neural networks comprising a frequency-domain deep neural network (DNN) including one or more gated recurrent unit (GRU) layers.
3. The one or more processors of claim 1 , wherein the one or more neural networks comprise a flow-based, generative network to perform audio generation, the flow-based, generative network caused to generate the one or more portions of speech at least by restoring primary audio modified in the one or more filtered digital signals.
4. The one or more processors of claim 1 , wherein the one or more circuits are further to use an auto-encoder network to enhance the one or more portions of speech generated by the one or more neural networks.
5. The one or more processors of claim 1 , wherein the one or more digital signals include a primary signal, corresponding to speech of a determined person, and noise including speech uttered by at least one other person.
6. The one or more processors of claim 1 , wherein the one or more circuits are further to provide the one or more filtered digital signals as input to the one or more neural networks as one or more spectrograms.
7. A system, comprising:
one or more processors to:
cause one or more digital signals to be filtered, where one or more portions of speech are removed from the one or more filtered digital signals;
generate, by one or more neural networks, the one or more portions of speech filtered from the one or more digital signals based, at least in part, on the one or more filtered digital signals; and
add the one or more portions of speech, generated by the one or more neural networks, to the one or more filtered digital signals.
8. The system of claim 7 , wherein the one or more processors are to cause one or more second neural networks to filter noise from the one or more digital signals, the one or more second neural networks comprising a frequency-domain deep neural network (DNN) including one or more gated recurrent unit (GRU) layers.
9. The system of claim 7 , wherein the one or more neural networks comprise a flow-based, generative network to perform audio generation, the flow-based, generative network caused to generate the one or more portions of speech at least by restoring primary audio modified in the one or more filtered digital signals.
10. The system of claim 7 , wherein the one or more processors are further to use an auto-encoder network to enhance the one or more portions of speech generated by the one or more neural networks.
11. The system of claim 7 , wherein the one or more digital signals include a primary signal, corresponding to speech of a determined person, and noise including speech uttered by at least one other person.
12. The system of claim 7 , wherein the one or more processors are further to provide the one or more filtered digital signals as input to the one or more neural networks as one or more spectrograms.
13. A method, comprising:
causing one or more digital signals to be filtered, where one or more portions of speech are removed from the one or more filtered digital signals;
generating, by one or more neural networks, the one or more portions of speech filtered from the one or more digital signals based, at least in part, on the one or more filtered digital signals; and
adding the one or more portions of speech, generated by the one or more neural networks, to the one or more filtered digital signals.
14. The method of claim 13 , wherein causing the one or more digital signals to be filtered comprises causing one or more second neural networks to filter noise from the one or more digital signals, the one or more second neural networks comprising a frequency-domain deep neural network (DNN) including one or more gated recurrent unit (GRU) layers.
15. The method of claim 13 , wherein the one or more neural networks comprise a flow-based, generative network to perform audio generation, and wherein generating, by the one or more neural networks, the one or more portions of speech comprises causing the flow-based, generative network to restore primary audio modified in the one or more filtered digital signals.
16. The method of claim 13 , further comprising using an auto-encoder network to enhance the one or more portions of speech generated by the one or more neural networks.
17. The method of claim 13 , wherein the one or more digital signals include a primary signal, corresponding to speech of a determined person, and noise including speech uttered by at least one other person.
18. The method of claim 13 , further comprising:
providing the one or more filtered digital signals as input to the one or more neural networks as one or more spectrograms.
19. A non-transitory computer-readable storage medium having stored thereon a set of instructions which, if performed by one or more processors, cause the one or more processors to at least:
cause one or more digital signals to be filtered, where one or more portions of speech are removed from the one or more filtered digital signals;
generate, by one or more neural networks, the one or more portions of speech filtered from the one or more digital signals based, at least in part, on the one or more filtered digital signals; and
add the one or more portions of speech, generated by the one or more neural networks, to the one or more filtered digital signals.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the set of instructions, if performed by the one or more processors, cause the one or more processors to cause one or more second neural networks to filter noise from the one or more digital signals, the one or more second neural networks comprising a frequency-domain deep neural network (DNN) including one or more gated recurrent unit (GRU) layers.
21. The non-transitory computer-readable storage medium of claim 19 , wherein the one or more neural networks comprise a flow-based, generative network to perform audio generation, and wherein the set of instructions which, if performed by the one or more processors, cause the one or more processors to generate, by the one or more neural networks, the one or more portions of speech further cause the one or more processors to cause the flow-based, generative network to restore primary audio modified in the one or more filtered digital signals.
22. The non-transitory computer-readable storage medium of claim 19 , wherein the set of instructions, if performed by the one or more processors, further cause the one or more processors to:
use an auto-encoder network to enhance the one or more portions of speech generated by the one or more neural networks.
23. The non-transitory computer-readable storage medium of claim 19 , wherein the one or more digital signals include a primary signal, corresponding to speech of a determined person, and noise including speech uttered by at least one other person.
24. The non-transitory computer-readable storage medium of claim 19 , wherein the set of instructions, if performed by the one or more processors, further cause the one or more processors to:
provide the one or more filtered digital signals as input to the one or more neural networks as one or more spectrograms.
25. An audio de-noising system, comprising:
one or more processors to:
cause one or more digital signals to be filtered, where one or more portions of speech are removed from the one or more filtered digital signals;
generate, by one or more neural networks, the one or more portions of speech filtered from the one or more digital signals based, at least in part, on the one or more filtered digital signals; and
add the one or more portions of speech, generated by the one or more neural networks, to the one or more filtered digital signals; and
memory for storing network parameters to be used by the one or more neural networks.
26. The audio de-noising system of claim 25 , wherein the one or more processors are to cause one or more second neural networks to filter noise from the one or more digital signals, the one or more second neural networks comprising a frequency-domain deep neural network (DNN) including one or more gated recurrent unit (GRU) layers.
27. The audio de-noising system of claim 25 , wherein the one or more neural networks comprise a flow-based, generative network to perform audio generation, the flow-based, generative network caused to generate the one or more portions of speech at least by restoring primary audio modified in the one or more filtered digital signals.
28. The audio de-noising system of claim 25 , wherein the one or more processors are further to use an auto-encoder network for enhancing the one or more portions of speech generated by the one or more neural networks.
29. The audio de-noising system of claim 25 , wherein the one or more digital signals include a primary signal, corresponding to speech of a determined person, and noise including speech uttered by at least one other person.
30. The audio de-noising system of claim 25 , wherein the one or more processors are further to provide the one or more filtered digital signals as input to the one or more neural networks as one or more spectrograms.
31. The one or more processors of claim 1 , wherein the one or more circuits are further to cause one or more neural networks to filter noise from the one or more filtered digital signals.
32. The one or more processors of claim 1 , wherein the one or more circuits are to cause one or more second neural networks to filter the one or more digital signals such that the one or more portions of speech are removed.
33. The one or more processors of claim 1 , wherein the one or more circuits are to cause the one or more neural networks to add the one or more portions of speech, generated by the one or more neural networks, to the one or more filtered digital signals.