Audio source separation

1. A method of extracting J audio sources from I audio channels, with I, J>1, wherein the audio channels comprise a plurality of clips, each clip comprising N frames, with N>1, wherein the I audio channels are representable as a channel matrix in a frequency domain, wherein the J audio sources are representable as a source matrix in the frequency domain, wherein the frequency domain is subdivided into F frequency bins, wherein the F frequency bins are grouped into F frequency bands, with F <F; wherein the method comprises, for a frame n of a current clip, for at least one frequency bin f, and for a current iteration,

updating a Wiener filter matrix based on

a mixing matrix, which is configured to provide an estimate of the channel matrix from the source matrix, and

a power matrix of the J audio source, which is indicative of a spectral power of the J audio sources;

wherein the Wiener filter matrix is configured to provide an estimate of the source matrix from the channel matrix; wherein the Wiener filter matrix is determined for each of the F frequency bins;

updating a cross-covariance matrix of the I audio channels and of the J audio sources and an auto-covariance matrix of the J audio sources, based on

the updated Wiener filter matrix; and

an auto-covariance matrix of the I audio channels; and

updating the mixing matrix and the power matrix based on

the updated cross-covariance matrix of the I audio channels and of the J audio sources, and/or

the updated auto-covariance matrix of the J audio sources; wherein the power matrix of the I audio sources is determined for the F frequency bands only.

2. The method of claim 1 , wherein the method comprises determining the auto-covariance matrix of the I audio channels for frame n of a current clip from frames of one or more previous clips and from frames of one or more future clips.

3. The method of claim 1 , wherein the method comprises determining the channel matrix by transforming the I audio channels from a time domain to the frequency domain, and optionally

wherein the channel matrix is determined using a short-term Fourier transform.

4. The method of claim 1 , wherein

the method comprises determining an estimate of the source matrix for the frame n of the current clip and for at least one frequency bin f as S fn =Ω fn X fn ;

S fn is an estimate of the source matrix;

Ω fn is the Wiener filter matrix; and

X fn is the channel matrix.

5. The method of claim 1 , wherein the method comprises performing the updating steps to determine the Wiener filter matrix, until a maximum number of iterations has been reached or until a convergence criteria with respect to the mixing matrix has been met.

6. The method of claim 1 , wherein the auto-covariance matrix of the I audio channels is determined for the F frequency bands only.

7. The method of claim 1 , wherein

the Wiener filter matrix is updated based on a noise power matrix comprising noise power terms; and

the noise power terms decrease with an increasing number of iterations.

8. The method of claim 1 , wherein

for the frame n of the current clip and for the frequency bin f lying within a frequency band f , the Wiener filter matrix is updated based on

Ω fn =Σ S, f n A fn H (A fn Σ S, f n A fn H +Σ B ) −1 for I<J, or based on

Ω f n =(A fn H Σ B −1 A fn +Σ S, f n −1 ) −1 A fn H Σ B −1 for I≥J;

Ω fn is the updated Wiener filter matrix;

Σ fn is the power matrix of the J audio sources;

A fn is the mixing matrix; and

Σ B is a noise power matrix.

9. The method of claim 1 , wherein the Wiener filter matrix is updated by applying an orthogonal constraint with regards to the J audio sources, and optionally

wherein the Wiener filter matrix is updated iteratively to reduce the power of non-diagonal terms of the auto-covariance matrix of the J audio sources.

10. The method of claim 9 , wherein

the Wiener filter matrix is updated iteratively using a gradient

(

Ω

f

_

⁢

n

⁢

R

XX

,

f

_

⁢

n

⁢

Ω

f

_

⁢

n

H

-

[

Ω

f

_

⁢

n

⁢

R

XX

,

f

_

⁢

n

⁢

Ω

f

_

⁢

n

H

]

D

)

⁢

Ω

f

_

⁢

n

⁢

R

XX

,

f

_

⁢

n



Ω

f

_

⁢

n



2

+

ϵ

;

Ω f n is the Wiener filter matrix for a frequency band f and for the frame n;

R XX, f n is the auto-covariance matrix of the I audio channels;

[ ] D is a diagonal matrix of a matrix included within the brackets, with all non-diagonal entries being set to zero; and

∈ is a real number.

11. The method of claim 1 , wherein

the cross-covariance matrix of the I audio channels and of the j audio sources is updated based on R XS, f n =R XX, f n Ω f n H ;

R XS, f n is the updated cross-covariance matrix of the I audio channels and of the J audio sources for a frequency band f and for the frame n;

Ω fn is the Wiener filter matrix; and

R XS, f n is the auto-covariance matrix of the I audio channels, and/or

wherein

the auto-covariance matrix of the J audio sources is updated based on R SS, f n =Ω f n R XX, f n Ω f n H ;

R SS, f n is the updated auto-covariance matrix of the J audio sources for a frequency band f and for the frame n;

Ω f n is the Wiener filter matrix; and

R XX, f n is the auto-covariance matrix of the I audio channels.

12. The method of claim 1 , wherein updating the mixing matrix comprises,

determining a frequency-independent auto-covariance matrix R SS,n of the J audio sources for the frame n, based on the auto-covariance matrices R SS, f n of the J audio sources for the frame n and for different frequency bins f or frequency bands f of the frequency domain; and

determining a frequency-independent cross-covariance matrix R XS,n of the I audio channels and of the J audio sources for the frame n based on the cross-covariance matrix R XS, f n of the I audio channels and of the J audio sources for the frame n and for different frequency bins f or frequency bands f of the frequency domain, and optionally

wherein

the mixing matrix is determined based on A n = R XS,n R SS,n −1 ,

A n is the frequency-independent mixing matrix for the frame n.

13. The method of claim 12 , wherein

the method comprises determining a frequency-dependent weighting term e fn based on the auto-covariance matrix R XX, f n of the I audio channels; and

the frequency-independent auto-covariance matrix R SS,n and the frequency-independent cross-covariance matrix R XS,n are determined based on the frequency-dependent weighting term e fn .

14. The method of claim 1 , wherein

updating the power matrix comprises determining an updated power matrix term (Σ s ) jj,fn for the j th audio source for the frequency bin f and for the frame n based on (Σ s ) jj,fn =(R SS, f n ) jj ; and

R SS, f n is the auto-covariance matrices of the J audio sources for the frame n and for a frequency band f which comprises the frequency bin f, and optionally

wherein

updating the power matrix comprises determining a spectral signature W and a temporal signature H for the J audio sources using a non-negative matrix factorization of the power matrix;

the spectral signature W and the temporal signature H for the j th audio source are determined based on the updated power matrix term (Σ s ) jj,fn for the j th audio source; and

updating the power matrix comprises determining a further updated power matrix term (Σ s ) jj,fn for the j th audio source based on (Σ s ) jj,fn =Σ k W j,fk H j,kn .

15. The method of claim 1 , wherein the method further comprises,

initializing the mixing matrix using a mixing matrix determined for a frame of a clip directly preceding the current clip; and

initializing the power matrix based on the auto-covariance matrix of the I audio channels for frame n of the current clip and based on the Wiener filter matrix determined for a frame of the clip directly preceding the current clip.