Isolating speech signals utilizing neural networks
View Patent ↗A speech signal isolation system configured to isolate and reconstruct a speech signal transmitted in an environment in which frequency components of the speech signal are masked by background noise. The speech signal isolation system obtains a noisy speech signal from an audio source. The noisy speech signal may then be fed through a neural network that has been trained to isolate and reconstruct a clean speech signal from against background noise. Once the noisy speech signal has been fed through the neural network, the speech signal isolation system generates an estimated speech signal with substantially reduced noise.
1. A speech signal isolation system for extracting a speech signal from background noise in an audio signal comprising:
a background noise estimation component adapted to estimate background noise intensity of an audio signal across a plurality of frequencies;
a neural network component adapted to extract a speech estimate signal from the background noise; and
a blending component for generating a reconstructed speech signal from the audio signal and the extracted speech, wherein the reconstructed speech signal comprises portions of the speech signal where an intensity of the speech signal is above the estimated background intensity level, portions of the extracted speech estimate signal where the intensity of the speech signal is below the estimated background intensity level, and a combination of the speech signal and the extracted speech estimate signal where the intensity of the speech signal is near the estimated background intensity level.
2. The system of claim 1 further comprising a frequency transform component for transforming said audio signal from a time-series signal to a frequency domain signal.
3. The system of claim 2 further comprising a compression component for generating a compressed audio signal having a reduced number of frequency subbands.
4. The system of claim 3 wherein the neural network has a first set of input nodes equal to the number of frequency subbands in the compressed audio signal, for receiving said compressed audio signal.
5. The system of claim 4 wherein the neural network includes a second set of input nodes equal to the number of frequency subbands, for receiving said background noise estimate.
6. The system of claim 4 wherein the neural network includes a second set of input nodes equal to the number of frequency subbands in the compressed audio signal for receiving the compressed audio signal from a previous time step.
7. The system of claim 4 wherein the neural network includes a second set of input nodes equal to the number of frequency subbands in the compressed audio signal, for receiving the output of the neural network from a previous time step.
8. The system of claim 4 wherein the neural network includes a second set of input nodes, for receiving an intermediate result from a previous time step.
9. A method of isolating a speech signal from an audio signal having a speech component and background noise, and the method comprising:
transforming a time-series audio signal into the frequency domain;
estimating the background noise in the audio signal across multiple frequency bands; extracting a speech signal estimate from the audio signal;
blending a portion of the speech signal estimate with a portion of the audio signal based on the background noise estimate to provide a reconstructed speech signal having reduced background noise, wherein the reconstructed speech signal comprises portions of the speech signal where an intensity of the speech signal is above an upper intensity threshold value which is greater than the estimated background intensity level, portions of the extracted speech estimate signal where the intensity of the speech signal is below a lower intensity threshold value which is near the estimated background intensity level, and a combination of the speech signal and the extracted speech estimate signal where the intensity of the speech signal is between the upper intensity threshold value and the lower intensity threshold value.
10. The method of claim 9 wherein extracting a speech signal estimate from the audio signal comprises assigning the audio signal as input to a neural network.
11. The method of claim 9 wherein combining the portions of the audio signal with portions of the speech signal estimate comprises weighting the audio signal and the speech signal estimate such that the speech signal estimate is given greater weight than the audio signal for portions of the audio signal having intensity values closer to the lower intensity threshold value, and greater weight to the audio signal than the speech signal estimate for those portions of the audio signal having intensity values closer to the upper intensity threshold value.
12. The method of claim 10 further comprising applying the background noise estimate to the neural network.
13. The method of claim 10 further comprising applying the speech signal estimate from a previous time step to the neural network.
14. The method of claim 10 further comprising applying an intermediate result of the speech signal estimate from a previous time step to the neural network.
15. The method of claim 10 further comprising applying the audio signal from a previous time step to the neural network.
16. A system for enhancing a speech signal comprising:
an audio signal source providing an audio time-series signal having both speech content and background noise;
a signal processor providing a frequency transform function for transforming the audio signal from the time-series domain to the frequency domain;
a background noise estimator;
a neural network; and
a signal combiner
said background noise estimator forming an estimate of the background noise in said audio signal, and said neural network extracting the speech signal estimate from said audio signal, and said signal combiner combining the speech signal estimate and the audio signal based on the background noise, estimate to produce a reconstituted speech signal having substantially reduced background noise, wherein the reconstructed speech signal comprises portions of the speech signal where an intensity of the speech signal is above the estimated background intensity level, portions of the extracted speech estimate signal where the intensity of the speech signal is below the estimated background intensity level, and a combination of the speech signal and that extracted speech estimate signal where the intensity of the speech signal is near the estimated background intensity level.
17. The system of claim 16 wherein the neural network comprises a first set of input nodes for receiving the audio signal.
18. The system of claim 17 wherein the neural network comprises a second set of input nodes for receiving the audio signal from a previous time step.
19. The system of claim 17 wherein the neural network comprises a second set of input nodes for receiving the background noise estimate.
20. The system of claim 17 wherein the neural network comprises a second set of input nodes for receiving the speech signal estimate from a previous time step.
21. The system of claim 17 wherein the neural network comprises a second set of input nodes for receiving an intermediate result from a previous time step.