Acoustic signal enhancement apparatus, method and program
Provided is an acoustic signal enhancement device, including a time-space covariance matrix estimation unit configured to estimate a time-space covariance matrix corresponding to a sound source, using a power of the sound source and an observation signal vector composed of an observation signal from a microphone. A reverberation suppression unit is configured to obtain a reverberation removal filter of the sound source using the time-space covariance matrix, and to generate a reverberation suppression signal vector corresponding to the observation signal for an emphasized sound of the sound source using the reverberation removal filter and the observation signal vector. A sound source separation unit is configured to obtain an emphatic sound of the sound source and the power of the sound source using the reverberation suppression signal vector.
1. An acoustic signal enhancement device, comprising:
processing circuitry configured to implement:
an input unit configured to receive, from a microphone m of a sound source n, an observation signal x m,t,f as input;
a time-space covariance matrix estimation unit configured to estimate a time-space covariance matrix R f (n) ,P f (n) corresponding to the sound source n, using a power λ t,f (n) of the sound source n and an observation signal vector X t,f composed of the observation signal x m,t,f from the microphone m, wherein t denotes a time frame number, f denotes a frequency number, N denotes the number of sound sources, M denotes the number of microphones, n is any number from 1 to N, and m is any number from 1 to M;
a reverberation suppression unit configured to obtain a reverberation removal filter G f (n) of the sound source n using the estimated time-space covariance matrix R f (n) , P f (n) , and to generate a reverberation suppression signal vector Z t,f (n) corresponding to the observation signal x m,t,f for an emphasized sound of the sound source n using the obtained reverberation removal filter G f (n) and the observation signal vector X t,f ;
a sound source separation unit configured to obtain an emphatic sound y t,f (n) of the sound source n and the power λ t,f (n) of the sound source n using the generated reverberation suppression signal vector Z t,f (n) ;
a control unit configured to control repeated processing of the time-space covariance matrix estimation unit, the reverberation suppression unit, and the sound source separation unit,
wherein the sound source separation unit is configured to repeatedly execute: (1) processing of obtaining a spatial covariance matrix Σ Z,f (n) corresponding to the sound source n using the generated reverberation suppression signal vector Z t,f (n) and the power λ t,f (n) of the sound source n, (2) processing of updating a separation filter Q f (n) corresponding to the sound source n using separation matrix W f =[Q f (1) , Q f (2) , . . . , Q f (N) ] T ∈C M×N and the obtained spatial covariance matrix Σ Z,f (n) , (3) processing of updating the emphatic sound y t,f (n) of the sound source n using the updated separation filter Q f (n) and the generated reverberation suppression signal vector Z t,f (n) and (4) processing of updating the power λ t,f (n) of the sound source n using the updated emphatic sound y t,f (n) , thereby finally obtaining the emphatic sound y t,f (n) of the sound source n; and
an output unit configured to convert the obtained emphatic sound y t,f (n) of the sound source n into output data and to output the output data, wherein the output data indicate emphasis based on at least a part of the emphatic sound y t,f (n) of the sound source n, and the output data further indicate suppressed reverberation of the at least a part of the emphatic sound y t,f (n) of the sound source n.
2. An acoustic signal enhancement method, comprising:
input operation by an input unit, by receiving, from a microphone m of a sound source n, an observation signal x m,t,f as input;
time-space covariance matrix estimation by a time-space covariance matrix estimation unit, by estimating a time-space covariance matrix R f (n) , P f (n) ) corresponding to the sound source n, using a power λ t,f (n) of the sound source n and an observation signal vector X t,f composed of the observation signal x m,t,f from the microphone m, wherein t denotes a time frame number, f denotes a frequency number, N denotes the number of sound sources, M denotes the number of microphones, n is any number from 1 to N, and m is any number from 1 to M;
reverberation suppression by a reverberation suppression unit, by obtaining a reverberation removal filter G f (n) of the sound source n using the estimated time-space covariance matrix R f (n) ,P f (n) , and generating a reverberation suppression signal vector Z t,f (n) corresponding to the observation signal x m,t,f for an emphasized sound of the sound source n using the obtained reverberation removal filter G f (n) and the observation signal vector X t,f ;
sound source separation by a sound source separation unit, by obtaining an emphatic sound y t,f (n) of the sound source n and the power λ t,f (n) of the sound source n using the generated reverberation suppression signal vector Z t,f (n) ;
by a control unit, controlling repeated processing of the time-space covariance matrix estimation, the reverberation suppression, and the sound source separation,
wherein the sound source separation unit is configured to repeatedly execute: (1) processing of obtaining a spatial covariance matrix Σ Z,f (n) corresponding to the sound source n using the generated reverberation suppression signal vector Z t,f (n) and the power λt,f (n) of the sound source n, (2) processing of updating a separation filter Q f (n) corresponding to the sound source n using separation matrix W f =[Q f (1) , Q f (2) , . . . , Q f (N) ] T ∈C M×N and the obtained spatial covariance matrix Σ Z,f (n) , (3) processing of updating the emphatic sound y t,f (n) of the sound source n using the updated separation filter Q f (n) and the generated reverberation suppression signal vector Z t,f (n) , and (4) processing of updating the power λ t,f (n) of the sound source n using the updated emphatic sound y t,f (n) , thereby finally obtaining the emphatic sound y t,f (n) of the sound source n; and
output by an output unit, by converting the obtained emphatic sound y t,f (n) of the sound source n into output data and to output the output data, wherein the output data indicate emphasis based on at least a part of the emphatic sound y t,f (n) of the sound source n, and the output data further indicate suppressed reverberation of the at least a part of the emphatic sound y t,f (n) of the sound source n.
3. A non-transitory computer readable medium that stores a program for causing a computer to perform as each step of the acoustic signal enhancement method according to claim 2 .