Separation of panned sources from generalized stereo backgrounds using minimal training
In an embodiment, a spatio-level filter (SLF) is created by obtaining a first set of samples from a plurality of target source level and spatial distributions in frequency subbands in a frequency domain, obtaining a second set of samples from a plurality of background level and spatial distributions in frequency subbands in a frequency domain, adding the first and second sets of samples to create a combined set of samples, detecting level and spatial parameters for each sample in the combined set of samples for each subband, within subbands, weighting the detected level and spatial parameters by their respective level and spatial distributions for the target source and backgrounds; storing the weighted level, spatial parameters and signal-to-noise ratio (SNR) within subbands for each sample in the combined set of samples in a table; and re-indexing the table by the weighted level and spatial parameters and subband.
1. A method comprising:
sampling, using one or more processors, a first spatial and level distribution of sound sources to obtain a first set of samples representing a target source;
sampling, using the one or more processors, a second spatial and level distribution of sound sources to obtain a second set of samples representing background sources;
obtaining, using the one or more processors, a frequency domain representation of the first set of samples in a plurality of frequency subbands;
obtaining, using the one or more processors, a frequency domain representation of the second set of samples in the plurality of frequency subbands;
pairwise adding, using the one or more processors, the frequency domain representations of the first and second sets of samples to create a plurality of training data sets;
detecting, using the one or more processors, respective parameter values for each of the training data sets for each frequency subband in the plurality of frequency subbands;
for each frequency subband of the plurality of frequency subbands, weighting, using the one or more processors, a contribution to a popularity array of each combination of the respective parameter values by the first and second spatial and level distributions;
storing, using the one or more processors, the popularity array and corresponding signal-to-noise ratio (SNR) values for the plurality of frequency subbands in a table, wherein each of the corresponding SNR values represents a level ratio of the target source to the background sources in a respective training data set; and
re-indexing, using the one or more processors, the table based on a Bayesian relationship between distributions of the respective parameter values and of the corresponding SNR values such that, for a given input set of parameter values, estimated SNR values associated with the given input set of parameter values for the plurality of frequency subbands are obtained from the re-indexed table via a lookup operation.
2. The method of claim 1 , further comprising:
smoothing data in the popularity array over one or more dimensions of the table.
3. The method of claim 1 , wherein the frequency domain representation is a short-time Fourier transform (STFT) domain representation.
4. The method of claim 1 , wherein spatial parameters represented by the respective parameter values include panning and a phase difference between two channels of a mixed audio signal corresponding to the plurality of training data sets.
5. The method of claim 1 , wherein the target source is amplitude panned using a constant power law.
6. A method comprising:
transforming, using one or more processors, one or more frames of a two-channel time domain audio signal into a time-frequency domain representation including a plurality of time-frequency tiles, wherein the time-frequency domain representation includes a plurality of frequency bins grouped into a plurality of subbands;
for each tile of the time-frequency domain representation:
calculating, using the one or more processors, a respective set of spatial parameter values and a respective level;
generating, using the one or more processors and a lookup table, a respective percentile signal-to-noise ratio (SNR) value for each frequency subband of the plurality of frequency subbands, wherein the lookup table is trained based on a Bayesian relationship between distributions of parameter values and corresponding SNR values such that, in response to a given input set of parameter values, the lookup table returns estimated SNR values associated with the given input set of parameter values for the plurality of frequency subbands, and wherein each of the corresponding SNR values represents a level ratio of the target source to background sources in a respective frequency subband of the plurality of frequency subbands;
generating, using the one or more processors, a softmask of fractional values based on the respective percentile SNR values, with each fractional value representing an audio portion of a corresponding subband attributed to the target source in a respective tile of the time-frequency domain representation; and
applying, using the one or more processors, the softmask of fractional values for to the plurality of frequency subbands in the respective tile to generate a modified time-frequency tile corresponding to the target source.
7. The method of claim 6 , further comprising:
transforming, using the one or more processors, the modified time-frequency tile into a corresponding time domain audio signal.
8. The method of claim 6 , wherein the transforming comprises applying a short-time frequency transform (STFT) to the two-channel time domain audio signal.
9. The method of claim 6 , wherein the plurality of frequency bins is grouped to form octave subbands or approximately octave subbands.
10. An apparatus comprising:
one or more processors; and
a memory storing instructions that when executed by the one or more processors, cause the apparatus to perform the method of claim 1 .