Source separation and remixing in signal processing

1 . A method of processing audio for source separation, the method comprising:

obtaining an audio signal including a mixture of speech content and noise content;

determining, from the audio signal, speech content;

determining, from the audio signal, stationary noise content;

determining, from the audio signal, non-speech content, wherein the stationary noise content is a true subset of the non-speech content;

determining, based on a difference between the stationary noise content and the non-speech content, a non-stationary noise content;

obtaining a set of weighting factors, the set comprising a weighting factor corresponding to each of said speech content, said stationary noise content, and said non-stationary noise content respectively; and

forming a processed audio signal based on a combination of the speech content, the stationary noise content, and the non-stationary noise content weighted with the respective weighting factor.

2 . The method according to claim 1 , wherein determining the stationary noise content comprises:

providing the audio signal to a stationary noise isolator model trained to predict a stationary noise mask for removing stationary noise content from the audio signal; and

determining the stationary noise content based on the stationary noise mask and the audio signal.

3 . The method according to claim 1 , wherein determining the non-speech content comprises:

providing the audio signal to a speech isolator model trained to predict a noise mask for removing non-speech content from the audio signal; and

determining non-speech content based on the noise mask and the audio signal.

4 . The method according to claim 1 , further comprising:

bandpass filtering the non-stationary noise content with a bandpass filter configured to isolate a noise object in the non-stationary noise content.

5 . The method according to claim 4 , further comprising:

bandpass filtering the non-stationary noise content with at least two different bandpass filters, each bandpass filter being configured to isolate a different noise object in the non-stationary noise.

6 . The method according to claim 4 , further comprising:

providing the audio signal to a noise object classifier model, the classifier model being trained to output a prediction of a noise object present in the audio signal;

providing a plurality bandpass filters, each configured to isolate a different noise object in the non-stationary noise; and

selecting the bandpass filter associated with the predicted noise object.

7 . The method according to claim 4 , wherein each bandpass filter has been obtained by:

collecting an example audio signal, the example audio signal comprising at least one example of a noise object;

determining the frequency distribution of the example audio signal; and

defining the bandpass filter based on the frequency distribution of the example audio signal.

8 . The method according to claim 4 , further comprising

smoothing the filtered non-stationary noise with a smoothing filter.

9 . The method according to claim 1 , wherein the weighting factors indicates boosting the non-stationary noise content with respect to the stationary noise content.

10 . The method according to claim 1 , further comprising:

providing at least two sets of weighting factors, each set of weighting factors being associated with a respective audio source type;

providing the audio signal to a classifier model, trained to output a prediction of a noise object present in the audio signal; and

wherein obtaining a set of weighting factors comprises:

selecting a set of said at least two sets, the selected set being associated with the predicted noise object.

11 . The method according to claim 1 , further comprising:

determining, based on the audio signal, at least one noise object, the noise object forming a true subset of the non-stationary noise content; and

wherein the set of weighting factors further comprises a noise object weighting factor for each noise object, and wherein said combination is further based on the noise object weighted with the noise object weighting factor.

12 . The method according to claim 11 , wherein determining at least one noise object comprises:

providing the audio signal to an object isolation model trained to predict a mask for separating the noise object from the audio signal; and

determining the noise object based on the audio signal and the mask for separating the noise object from the audio signal.

13 . The method according to claim 11 , further comprising:

providing a plurality of trained object isolation models, each model trained to predict a mask for separating a different noise object from an audio signal;

providing the audio signal to a classifier model, trained to output a predicted noise object present in the audio signal;

selecting, from said plurality of trained object isolation models, the trained object isolation model associated with the predicted noise object; and

providing the audio signal to the selected object isolation model to predict a mask for separating the predicted noise object from the audio signal.

14 . The method according to claim 1 , wherein the difference between the stationary noise content and the non-speech content is calculated such that the non-stationary noise is independent from the stationary noise.

15 . The method according to claim 1 , wherein determining the non-stationary noise content comprises:

capturing all content that is neither the stationary noise content nor the speech content.

16 . The method according to claim 1 , further comprising:

determining, based on applying a mask to the audio signal to remove the stationary noise content, a residual content that is all content of the audio signal excluding the stationary noise content; and

obtaining a second set of weighting factors, the second set comprising a weighting factor corresponding to each of the residual content, the non-speech content, and the stationary noise content, respectively, wherein a mapping exists between the set of weighting factors and the second set of weighting factors; and

forming the processed audio signal based on a combination of the residual content, the non-speech content, and the stationary noise content.

17 . An audio processing system, the audio processing system comprising:

an audio content separation unit, the audio content separation unit being configured to:

obtain an audio signal, the audio signal including a mixture of speech content and noise content,

determine, from the audio signal, speech content,

determine, from the audio signal, stationary noise content,

determine, from the audio signal, non-speech content, wherein the stationary noise content is a true subset of the non-speech content, and

determine, based on a difference between the stationary noise content and the non-speech content, a non-stationary noise content,

the audio processing system further comprising a mixing unit configured to:

obtain a set of weighting factors, the set comprising a weighting factor corresponding to each of said speech content, said stationary noise content, and said non-stationary noise content respectively, and

form a processed audio signal based on a combination of the speech content, the stationary noise content, and the non-stationary noise content weighted with the respective weighting factor.

18 . A non-transitory computer-readable medium storing instructions that, upon execution by one or more processors, cause the one or more processor to perform the method of claim 1 .