Signal processing device, signal processing method, and computer program product

1. A signal processing device comprising:

a microprocessor configured to operate as at least:

an acquirer configured to acquire a first acoustic signal and a second acoustic signal;

a first background sound calculator configured to calculate a first background sound signal in which a speech signal is removed, based on the first acoustic signal and the second acoustic signal;

a first signal generator configured to generate a first reference signal from at least one of the first acoustic signal and the second acoustic signal;

an extractor configured to extract a second background sound signal by removing a speech signal from the first reference signal;

a similarity calculator configured to calculate a first similarity indicating a degree of similarity between feature data of the first background sound signal and feature data of the second background sound signal; and

a mixer configured to calculate a weighted sum of the first background sound signal and the second background sound signal in such a way that a greater weight is given to the first background sound signal as the first similarity is higher and a greater weight is given to the second background sound signal as the first similarity is lower.

2. The device according to claim 1 , wherein the first background sound calculator is configured to calculate a first background sound signal that is a difference signal between the first acoustic signal and the second acoustic signal.

3. The device according to claim 1 , wherein the first signal generator is configured to generate a first reference signal that is one of the first acoustic signal, the second acoustic signal, and a weighted sum of the first acoustic signal and the second acoustic signal.

4. The device according to claim 1 , wherein

the extractor is configured to further extract a speech signal from the first reference signal, and

the mixer is configured to calculate a weighted sum of the first background sound signal, the second background sound signal, and the extracted speech signal.

5. The device according to claim 4 , wherein the microprocessor is further configured to operate as a third background sound generator configured to generate a third background sound signal by further removing a speech signal from the first background sound signal, and

the mixer is configured to calculate a weighted sum of the third background sound signal, the second background sound signal, and the extracted speech signal.

6. The device according to claim 5 , wherein the microprocessor is further configured to operate as a setter configured to set sound source information indicating a sound source on which importance is placed in an output,

the extractor is configured to extract a speech signal from the first reference signal according to the sound source information and the first similarity,

the third background sound generator is configured to generate the third background sound signal according to the sound source information and the first similarity, and

the mixer is configured to

give a greater weight to the extracted speech signal when the sound source information indicates that importance is placed on speech, and

give greater weights to the third background signal and the second background sound signal when the sound source information indicates that importance is placed on a background sound.

7. The device according to claim 6 , wherein the extractor is configured to switch to simpler processing when the sound source information indicates that importance is placed on a background sound and when the first similarity is equal to or higher than a threshold.

8. The device according to claim 6 , wherein the third background sound generator is configured to switch to simpler processing when the sound source information indicates that importance is placed on speech or when the first similarity is smaller than a threshold.

9. The device according to claim 6 , wherein the third background sound generator is configured to generate the first background sound signal as the third background sound signal when the sound source information indicates that importance is placed on speech or when the first similarity is smaller than a threshold.

10. The device according to claim 1 , wherein

the similarity calculator is configured to further calculate a second similarity indicating a degree of similarity between the feature data of the first background sound signal and feature data of the first reference signal, and

the microprocessor is further configured to operate as a corrector configured to correct the first similarity according to the second similarity.

11. The device according to claim 10 , wherein

the similarity calculator further includes a similarity acquirer configured to acquire an already calculated similarity that is the first similarity calculated at a first time, and

the corrector is configured to make an amount of correction of the first similarity calculated at a second time later than the first time greater as the already calculated similarity is lower.

12. The device according to claim 1 , wherein

the similarity calculator includes

a non-reliability calculator configured to calculate a non-reliability indicating a degree of likelihood of the first background sound signal being a noise, and

a corrector configured to correct the first similarity according to the non-reliability.

13. The device according claim 1 , wherein

the similarity calculator includes

a level calculator configured to calculate a first background sound signal level that is a amplitude of the first background sound signal within a unit time, and a second background sound signal level that is a amplitude of the second background sound signal within the unit time, and

a similarity generator configured to make the first similarity higher as a ratio of the first background sound signal level to the second background sound signal level is greater.

14. The signal processing device according to claim 1 , wherein

the similarity calculator includes a second signal generator configured to generate a third reference signal that is a weighted sum of the first reference signal and the second background sound signal, and

the similarity calculator is configured to calculate the first similarity according to a degree of similarity between the feature data of the first background sound signal and feature data of the third reference signal.

15. The device according to claim 14 , wherein

the similarity calculator further includes a similarity acquirer configured to acquire an already calculated similarity that is the first similarity calculated at a first time, and

the second signal generator is configured to make a weight to be given to the second background sound signal greater as the already calculated similarity is higher.

16. The device according to claim 14 , wherein

the similarity calculator includes

a level calculator configured to calculate a first background sound signal level that is a amplitude of the first background sound signal within a unit time, and a third reference signal level that is a amplitude of the third reference signal within the unit time, and

a similarity generator configured to make the first similarity higher as a ratio of the first background sound signal to the third reference signal level is greater.

17. A signal processing method comprising:

acquiring a first acoustic signal and a second acoustic signal;

calculating a first background sound signal in which a speech signal is removed, based on the first acoustic signal and the second acoustic signal;

generating a first reference signal from at least one of the first acoustic signal and the second acoustic signal;

extracting a second background sound signal by removing a speech signal from the first reference signal;

calculating a first similarity indicating a degree of similarity between feature data of the first background sound signal and feature data of the second background sound signal; and

calculating a weighted sum of the first background sound signal and the second background sound signal in such a way that a greater weight is given to the first background sound signal as the first similarity is higher and a greater weight is given to the second background sound signal as the first similarity is lower.

18. A computer program product comprising a non-transitory computer-readable medium containing a program executed by a computer, the program causing the computer to execute at least:

acquiring a first acoustic signal and a second acoustic signal;

calculating a first background sound signal in which a speech signal is removed, based on the first acoustic signal and the second acoustic signal;

generating a first reference signal from at least one of the first acoustic signal and the second acoustic signal;

extracting a second background sound signal by removing a speech signal, from the first reference signal;

calculating a first similarity indicating a degree of similarity between feature data of the first background sound signal and feature data of the second background sound signal; and

calculating a weighted sum of the first background sound signal and the second background sound signal in such a way that a greater weight is given to the first background sound signal as the first similarity is higher and a greater weight is given to the second background sound signal as the first similarity is lower.