Acoustic signal enhancement apparatus, method and program
An acoustic signal enhancement device includes: a spatiotemporal covariance matrix estimation unit 2 configured to estimate spatiotemporal covariance matrices R f (j) and P f (j) ; a reverberation suppression unit 3 configured to obtain a reverberation suppression filter G f (j) of the sound source j using the estimated spatiotemporal covariance matrices R f (j) and P f (j) for each sound source j and to generate a reverberation suppression signal vector using the obtained reverberation suppression filter G f (j) and the observation signal vector X t,f ; a sound source separation unit 4 configured to obtain an enhanced sound y t,f (j) of the sound source j and power of the sound source j using the generated reverberation suppression signal vector for each sound source j (where 1≤j≤J) corresponding to the target sound; and a control unit 5 configured to perform control such that processes of these units are repeatedly performed.
1 . An acoustic signal enhancement device comprising:
processing circuitry configured to:
estimate spatiotemporal covariance matrices R f (j) and P f (j) using power λ t (j) of a sound source j and an observation signal vector X t,f formed from an observation signal X t,f (m) of a microphone m for each sound source j when t is a time frame number, f is a frequency number, M is the number of microphones, m=1, . . . , M, there are a target sound and noise in the sound source, J is the number of target sounds, M>J, j=1, . . . , J+1, j of 1≤j≤J indicates a sound source corresponding to the target sound, and J+1 indicates a sound source corresponding to the noise;
obtain a reverberation suppression filter G f (j) of the sound source j using the estimated spatiotemporal covariance matrices R f (j) and P f (j) for each sound source j and to generate a reverberation suppression signal vector using the obtained reverberation suppression filter G f (j) and the observation signal vectors X t,f ,
obtain an enhanced sound y t,f (j) of the sound source j and power of the sound source j using the generated reverberation suppression signal vector for each sound source j (where 1≤j≤J) corresponding to the target sound; and
perform control such that processes of the processing circuitry are repeatedly performed.
2 . The acoustic signal enhancement device according to claim 1 , wherein
the processing circuitry further configured to obtain the reverberation suppression filter G f (j) of the sound source j using the estimated spatiotemporal covariance matrices R f (j) and P f (j) for each sound source j, and generate a reverberation suppression signal vector Z t,f (1) corresponding to an observation signal X t,f (m) regarding an enhanced sound of the sound source j using the obtained reverberation suppression filter G f (j) and the observation signal vector X t,f , and
the processing circuitry further configured to obtain the enhanced sound y t,f (j) of the sound source j and the power of the sound source j using the generated reverberation suppression signal vector Z t,f (j) for each sound source j (where 1≤j≤J) corresponding to the target sound.
3 . The acoustic signal enhancement device according to claim 2 , wherein
the processing circuitry further configured to obtain the enhanced sound y t,f (j) of the sound source j by repeating (1) a process of obtaining a spatial covariance matrix Σ f (j) corresponding to the sound source j using the generated reverberation suppression signal vector Z t,f (j) and the power of the sound source j, (2) a process of updating a separation filter Q f (f) corresponding to the sound source j using the obtained spatial covariance matrix Σ f (j) , updating the enhanced sound y t,f (j) of the sound source j using the updated separation filter Q f (j) and the generated reverberation suppression signal vector Z t,f (j) , and updating the power of the sound source j using the updated enhanced sound y t,f (j) , and (3) a process of updating the noise separation matrix Q N,f using the updated separation filter Q f (j) .
4 . The acoustic signal enhancement device according to claim 1 , wherein
the processing circuitry further configured to obtain the reverberation suppression filter G f (j) of the sound source j using the estimated spatiotemporal covariance matrices R f (j) and P f (j) for each sound source j, obtain a reverberation suppression filter G f common to all sound sources from the obtained reverberation suppression filter G f (j) , and generate a reverberation suppression signal vector Z t,f formed from a reverberation suppression signal z t,f (m) corresponding to an observation signal x t,f (m) using the obtained reverberation suppression filter Grand the observation signal vector X t,f , and
the processing circuitry further configured to obtain the enhanced sound y t,f (j) of the sound source j and the power of the sound source j using the generated reverberation suppression signal vector Z t,f for each sound source j (where 1≤j≤J) corresponding to the target sound.
5 . The acoustic signal enhancement device according to claim 4 , wherein
the processing circuitry further configured to finally obtain the enhanced sound y t,f (j) of the sound source j by repeating (1) a process of obtaining a spatial covariance matrix Σ f (j) corresponding to the sound source j using the generated reverberation suppression signal vector Z t,f and the power of the sound source j, (2) a process of updating a separation filter Q f (j) corresponding to the sound source j using the obtained spatial covariance matrix Σ f (j) , updating the enhanced sound y t,f (j) of the sound source j using the updated separation filter Q f (j) and the generated reverberation suppression signal vector Z t,f , and updating the power of the sound source j using the updated enhanced sound y t,f (j) , and (3) a process of updating the noise separation matrix Q N,f using the updated separation filter Q f (j) .
6 . An acoustic signal enhancement method comprising:
a spatiotemporal covariance matrix estimation step of estimating spatiotemporal covariance matrices R f (j) and P f (j) using power of a sound source j and an observation signal vector X t,f formed from an observation signal x t,f (m) of a microphone m for each sound source j by a spatiotemporal covariance matrix estimation unit when t is a time frame number, f is a frequency number, M is the number of microphones, m=1, . . . , M, there are a target sound and noise in the sound source, J is the number of target sounds, M>J, j=1, . . . , J+1, j of 1≤j≤J indicates a sound source corresponding to the target sound, and J+1 indicates a sound source corresponding to the noise;
a reverberation suppression step of obtaining a reverberation suppression filter G f (j) of the sound source j using the estimated spatiotemporal covariance matrices R f (j) and P f (j) for each sound source j and generating a reverberation suppression signal vector using the obtained reverberation suppression filter G f (j) and the observation signal vectors X t,f by a reverberation suppression unit;
a sound source separation step of obtaining an enhanced sound y t,f (j) of the sound source j and power of the sound source j using the generated reverberation suppression signal vector for each sound source j (where 1≤j≤J) corresponding to the target sound by a sound source separation unit; and
a control step of performing control by a control unit such that a process of the spatiotemporal covariance matrix estimation unit, a process of the reverberation suppression unit, and a process of the sound source separation unit are repeatedly performed.
7 . A non-transitory computer-readable recording medium storing a computer-executable program instructions that when executed by a processor cause causing a computer to execute operations comprising:
a spatiotemporal covariance matrix estimation step of estimating spatiotemporal covariance matrices R f (j) and P f (j) using power of a sound source j and an observation signal vector X t,f formed from an observation signal x t,f (m) of a microphone m for each sound source i by a spatiotemporal covariance matrix estimation unit when t is a time frame number, f is a frequency number, M is the number of microphones, m=1, . . . , M, there are a target sound and noise in the sound source, J is the number of target sounds, M>J, j=1, . . . , J+1, j of 1≤j≤J indicates a sound source corresponding to the target sound, and J+1 indicates a sound source corresponding to the noise;
a reverberation suppression step of obtaining a reverberation suppression filter G f (j) of the sound source j using the estimated spatiotemporal covariance matrices R f (j) and P f (j) for each sound source j and generating a reverberation suppression signal vector using the obtained reverberation suppression filter G f (j) and the observation signal vectors X t,f by a reverberation suppression unit;
a sound source separation step of obtaining an enhanced sound y t,f (j) of the sound source j and power of the sound source j using the generated reverberation suppression signal vector for each sound source j (where 1<j≤J) corresponding to the target sound by a sound source separation unit; and
a control step of performing control by a control unit such that a process of the spatiotemporal covariance matrix estimation unit, a process of the reverberation suppression unit, and a process of the sound source separation unit are repeatedly performed.