Device for assisting two-way conversation and method for assisting two-way conversation

1. A two-way conversation assisting device for amplifying and assisting a two-way conversation,

the two-way conversation assisting device comprising:

a first microphone that enters a first voice;

a first loudspeaker that outputs the first voice;

a second microphone that enters a second voice;

a second loudspeaker that outputs the second voice; and

a first echo and crosstalk canceller that estimates and calculates, using an input signal into the second loudspeaker, a first interference signal indicative of degrees of a first acoustic echo caused when the second voice output from the second loudspeaker enters into the first microphone and first acoustic crosstalk caused when the second voice enters into the first microphone, and that removes the calculated first interference signal from an output signal of the first microphone.

2. The two-way conversation assisting device according to claim 1 , further comprising

a first acoustic feedback canceller that estimates and calculates a first acoustic feedback signal indicative of a degree of first acoustic feedback caused when the second voice output from the second loudspeaker enters into the second microphone, and that removes the calculated first acoustic feedback signal from an output signal of the second microphone.

3. The two-way conversation assisting device according to claim 2 , further comprising:

a second acoustic feedback canceller that estimates and calculates a second acoustic feedback signal indicative of a degree of second acoustic feedback caused when the first voice output from the first loudspeaker enters into the first microphone, and that removes the calculated second acoustic feedback signal from the output signal of the first microphone; and

a second echo and crosstalk canceller that estimates and calculates, using an input signal into the first loudspeaker, a second interference signal indicative of degrees of a second acoustic echo caused when the first voice output from the first loudspeaker enters into the second microphone and second acoustic crosstalk caused when the first voice enters into the second microphone, and that removes the calculated second interference signal from the output signal of the second microphone.

4. The two-way conversation assisting device according to claim 3 , further comprising:

a first acoustic canceller that estimates and calculates, using an acoustic signal, a first acoustic echo signal indicative of a degree of a first acoustic echo caused when a sound output from the second loudspeaker enters into the first microphone, and that removes the calculated first acoustic echo signal from the output signal of the first microphone; and

a second acoustic canceller that estimates and calculates, using the acoustic signal, a second acoustic echo signal indicative of a degree of a second acoustic echo caused when a sound output from the first loudspeaker enters into the second microphone, and that removes the calculated second acoustic echo signal from the output signal of the second microphone.

5. The two-way conversation assisting device according to claim 4 , wherein

the first acoustic canceller includes

a fifth transfer function storage circuit that stores a transfer function estimated as a transfer function with respect to the first acoustic echo,

a fifth storage circuit that stores the acoustic signal,

a fifth convolution arithmetic operation unit that performs a convolution with the acoustic signal stored in the fifth storage circuit and the transfer function stored in the fifth transfer function storage circuit to generate the calculated first acoustic echo signal,

a fifth subtractor that removes from the output signal of the first microphone the calculated first acoustic echo signal output from the fifth convolution arithmetic operation unit, and that outputs an output signal of the first acoustic canceller, and

a fifth transfer function update circuit that updates the transfer function stored in the fifth transfer function storage circuit based on the output signal of the fifth subtractor and the acoustic signal stored in the fifth storage circuit, and

the second acoustic canceller includes

a sixth transfer function storage circuit that stores a transfer function estimated as a transfer function with respect to the second acoustic echo,

a sixth storage circuit that stores the acoustic signal,

a sixth convolution arithmetic operation unit that performs a convolution with the acoustic signal stored in the sixth storage circuit and the transfer function stored in the sixth transfer function storage circuit to generate the calculated second acoustic echo signal,

a sixth subtractor that removes from the output signal of the second microphone the calculated second acoustic echo signal output from the sixth convolution arithmetic operation unit, and that outputs an output signal of the second acoustic canceller, and

a sixth transfer function update circuit that updates the transfer function stored in the sixth transfer function storage circuit based on the output signal of the sixth subtractor and the acoustic signal stored in the sixth storage circuit.

6. The two-way conversation assisting device according to claim 5 , wherein

the first acoustic canceller further includes a first adder that adds the acoustic signal to the input signal into the second loudspeaker, and

the second acoustic canceller further includes a second adder that adds the acoustic signal to the input signal into the first loudspeaker.

7. The two-way conversation assisting device according to claim 3 , wherein

for the first voice uttered at a certain time, a time when the output signal of the first microphone is input into the second echo and crosstalk canceller is identical to or earlier than a time when the first voice enters into the second microphone, and

for the second voice uttered at a certain time, a time when the output signal of the second microphone is input into the first echo and crosstalk canceller is identical to or earlier than a time when the second voice enters into the first microphone.

8. The two-way conversation assisting device according to claim 3 , wherein

the first acoustic feedback canceller includes

a first transfer function storage circuit that stores a transfer function estimated as a transfer function with respect to the first acoustic feedback,

a first delay device that delays an output signal of the first acoustic feedback canceller,

a first storage circuit that stores a signal output from the first delay device,

a first convolution arithmetic operation unit that performs a convolution with the signal stored in the first storage circuit and the transfer function stored in the first transfer function storage circuit to generate the calculated first acoustic feedback signal,

a first subtractor that removes from the output signal of the second microphone the first acoustic feedback signal output from the first convolution arithmetic operation unit, and that outputs the output signal of the first acoustic feedback canceller, and

a first transfer function update circuit that updates the transfer function stored in the first transfer function storage circuit based on the output signal of the first subtractor and the signal stored in the first storage circuit.

9. The two-way conversation assisting device according to claim 8 , wherein

the first echo and crosstalk canceller includes

a second transfer function storage circuit that stores a transfer function estimated as a transfer function with respect to the first acoustic echo and the first acoustic crosstalk combined to each other,

a second storage circuit that stores the output signal of the second microphone,

a second convolution arithmetic operation unit that performs a convolution with the output signal stored in the second storage circuit and the transfer function stored in the second transfer function storage circuit to generate the calculated first interference signal,

a second subtractor that removes from the output signal of the calculated first microphone the first interference signal output from the second convolution arithmetic operation unit, and that outputs an output signal of the first echo and crosstalk canceller, and

a second transfer function update circuit that updates the transfer function stored in the second transfer function storage circuit based on the output signal of the second subtractor and the output signal stored in the second storage circuit.

10. The two-way conversation assisting device according to claim 9 , wherein

the second acoustic feedback canceller includes

a third transfer function storage circuit that stores a transfer function estimated as a transfer function with respect to the second acoustic feedback,

a second delay device that delays an output signal of the second acoustic feedback canceller,

a third storage circuit that stores a signal output from the second delay device,

a third convolution arithmetic operation unit that performs a convolution with the signal stored in the third storage circuit and the transfer function stored in the third transfer function storage circuit to generate the second acoustic feedback signal,

a third subtractor that removes from the output signal of the first microphone the calculated second acoustic feedback signal output from the third convolution arithmetic operation unit, and that outputs the output signal of the second acoustic feedback canceller, and

a third transfer function update circuit that updates the transfer function stored in the third transfer function storage circuit based on the output signal of the third subtractor and the signal stored in the third storage circuit.

11. The two-way conversation assisting device according to claim 10 , wherein

the second echo and crosstalk canceller includes

a fourth transfer function storage circuit that stores a transfer function estimated as a transfer function with respect to the second acoustic echo and the second acoustic crosstalk combined to each other,

a fourth storage circuit that stores the output signal of the first microphone,

a fourth convolution arithmetic operation unit that performs a convolution with the output signal stored in the fourth storage circuit and the transfer function stored in the fourth transfer function storage circuit to generate the calculated second interference signal,

a fourth subtractor that removes from an output signal of the second microphone the calculated interference signal output from the fourth convolution arithmetic operation unit, and that outputs an output signal of the second echo and crosstalk canceller, and

a fourth transfer function update circuit that updates the transfer function stored in the fourth transfer function storage circuit based on the output signal of the fourth subtractor and the output signal stored in the fourth storage circuit.

12. The two-way conversation assisting device according to claim 11 , wherein

the first transfer function update circuit uses an independent component analysis to update the transfer function stored in the first transfer function storage circuit based on the signal of the first subtractor and the signal stored in the first storage circuit such that the signal of the first subtractor and the signal stored in the first storage circuit are independent from each other,

the second transfer function update circuit uses an independent component analysis to update the transfer function stored in the second transfer function storage circuit based on the output signal of the second subtractor and the output signal stored in the second storage circuit such that the output signal of the second subtractor and the output signal stored in the second storage circuit are independent from each other,

the third transfer function update circuit uses an independent component analysis to update the transfer function stored in the third transfer function storage circuit based on the output signal of the third subtractor and the signal stored in the third storage circuit such that the output signal of the third subtractor and the signal stored in the third storage circuit are independent from each other, and

the fourth transfer function update circuit uses an independent component analysis to update the transfer function stored in the fourth transfer function storage circuit based on the signal of the fourth subtractor and the output signal stored in the fourth storage circuit such that the signal of the fourth subtractor and the output signal stored in the fourth storage circuit are independent from each other.

13. The two-way conversation assisting device according to claim 12 , wherein

the first transfer function update circuit performs nonlinear processing using a nonlinear function on the output signal of the first subtractor, performs a multiplication on an obtained result with the signal stored in the first storage circuit and a first step size parameter for controlling a learning speed in estimating the transfer function with respect to the first acoustic feedback to calculate a first update coefficient, and adds the calculated first update coefficient to the transfer function stored in the first transfer function storage circuit for updating,

the second transfer function update circuit performs nonlinear processing using a nonlinear function on the output signal of the second subtractor, performs a multiplication on an obtained result with the signal stored in the second storage circuit and a second step size parameter for controlling a learning speed in estimating the transfer function with respect to the first acoustic echo and the first acoustic crosstalk combined to each other to calculate a second update coefficient, and adds the calculated second update coefficient to the transfer function stored in the second transfer function storage circuit for updating,

the third transfer function update circuit performs nonlinear processing using a nonlinear function on the output signal of the third subtractor, performs a multiplication on an obtained result with the signal stored in the third storage circuit and a third step size parameter for controlling a learning speed in estimating the transfer function with respect to the second acoustic feedback to calculate a third update coefficient, and adds the calculated third update coefficient to the transfer function stored in the third transfer function storage circuit for updating, and

the fourth transfer function update circuit performs nonlinear processing using a nonlinear function on the output signal of the fourth subtractor, performs a multiplication on an obtained result with the signal stored in the fourth storage circuit and a fourth step size parameter for controlling a learning speed in estimating the transfer function with respect to the second acoustic echo and the second acoustic crosstalk combined to each other to calculate a fourth update coefficient, and adds the calculated fourth update coefficient to the transfer function stored in the fourth transfer function storage circuit for updating.

14. The two-way conversation assisting device according to claim 13 , wherein

the nonlinear function used in each of the first transfer function update circuit, the second transfer function update circuit, the third transfer function update circuit, and the fourth transfer function update circuit is a sigmoid function, a hyperbolic tangent function, a rectified linear function, or a sign function.

15. The two-way conversation assisting device according to claim 1 , further comprising

a second echo and crosstalk canceller that estimates and calculates, using an input signal into the first loudspeaker, a second interference signal indicative of degrees of a second acoustic echo caused when the first voice output from the first loudspeaker enters into the second microphone and second acoustic crosstalk caused when the first voice enters into the second microphone, and that removes the calculated second interference signal from an output signal of the second microphone.

16. The two-way conversation assisting device according to claim 1 , wherein the output signal of the first microphone removes both the first acoustic echo and the first acoustic crosstalk together.

17. A two-way conversation assisting method for amplifying and assisting a two-way conversation,

the two-way conversation assisting method being used in a conversation system including

a first microphone that enters a first voice,

a first loudspeaker that outputs the first voice,

a second microphone that enters a second voice, and

a second loudspeaker that outputs the second voice,

the two-way conversation assisting method comprising

a first echo and crosstalk estimating and calculating, using an input signal into the second loudspeaker, a first interference signal indicative of degrees of a first acoustic echo caused when the second voice output from the second loudspeaker enters into the first microphone and first acoustic crosstalk caused when the second voice enters into the first microphone, and removing the calculated first interference signal from an output signal of the first microphone.

18. The two-way conversation assisting method according to claim 17 , further comprising

a first acoustic feedback estimating and calculating a first acoustic feedback signal indicative of a degree of first acoustic feedback caused when the second voice output from the second loudspeaker enters into the second microphone, and removing the calculated first acoustic feedback signal from an output signal of the second microphone.

19. The two-way conversation assisting method according to claim 18 , further comprising:

a second acoustic feedback estimating and calculating a second acoustic feedback signal indicative of a degree of second acoustic feedback caused when the first voice output from the first loudspeaker enters into the first microphone, and removing the calculated second acoustic feedback signal from the output signal of the first microphone; and

a second echo and crosstalk estimating and calculating, using an input signal into the first loudspeaker, a second interference signal indicative of degrees of a second acoustic echo caused when the first voice output from the first loudspeaker enters into the second microphone and second acoustic crosstalk caused when the first voice enters into the second microphone, and removing the calculated second interference signal from the output signal of the second microphone.

20. The two-way conversation assisting method according to claim 17 , further comprising

a second echo and crosstalk estimating and calculating, using an input signal into the first loudspeaker, a second interference signal indicative of degrees of a second acoustic echo caused when the first voice output from the first loudspeaker enters into the second microphone and second acoustic crosstalk caused when the first voice enters into the second microphone, and removing the calculated second interference signal from an output signal of the second microphone.

21. The two-way conversation assisting method according to claim 19 , further comprising:

a first acoustic estimating and calculating, using an acoustic signal, a first acoustic echo signal indicative of a degree of a first acoustic echo caused when a sound output from the second loudspeaker enters into the first microphone, and removing the calculated first acoustic echo signal from the output signal of the first microphone; and

a second acoustic estimating and calculating, using the acoustic signal, a second acoustic echo signal indicative of a degree of a second acoustic echo caused when a sound output from the first loudspeaker enters into the second microphone, and removing the calculated second acoustic echo signal from the output signal of the second microphone.

22. The two-way conversation assisting method according to claim 17 , wherein the output signal of the first microphone removes both the first acoustic echo and the first acoustic crosstalk together.