Information presentation device associated with sound source separation

1. An information presentation device comprising:

a processor;

an image display unit; and

an operation input unit receiving an operation by a user, wherein

the processor is programmed with instructions that, when executed, cause the processor at least to:

receive an audio signal and an image signal;

make the display unit to display an image indicated by the image signal;

estimate direction information for each sound source based on the audio signal;

separate the audio signal to sound-source-classified audio signals for each sound source;

generate coordinate designation information indicating a part of a region of the image based on the operation by the user which is input to the operation input unit; and

select a sound-source-classified audio signal of a sound source associated with a coordinate which is included in a region indicated by the coordinate designation information, and which corresponds to the direction information.

2. The information presentation device according to claim 1 , wherein the processor is further programmed with instructions that, when executed, cause the processor at least to:

detect whether or not the sound sources corresponding to the sound-source-classified audio signals that are in an utterance state;

generate display information indicating an utterance state on a coordinate corresponding to the direction information of the sound sources in which the utterance state is detected; and

make the display unit to display an image indicated by the display information while being superimposed over the image.

3. The information presentation device according to claim 1 , wherein the processor is further programmed with instructions that, when executed, cause the processor at least to:

recognize an utterance content based on the sound-source-classified audio signals;

generate display information indicating the utterance content at a coordinate corresponding to the direction information of the sound sources corresponding to the sound-source-classified audio signal; and

make the display unit to display an image indicated by the display information while being superimposed over the image.

4. An information presentation device, comprising:

a processor; and

an operation input unit receiving an operation by a user, wherein

the processor is programmed with instructions that, when executed, cause the processor at least to:

receive an image signal, direction information for each sound source, and a sound-source-classified audio signal for each sound source;

generate coordinate designation information indicating a part of a region of the image based on the operation by the user which is input to the operation input unit; and

select sound-source-classified audio signal of a sound source associated with a coordinate which is included in a region indicated by the coordinate designation information, and which corresponds to the direction information.

5. An information transmission system comprising:

an information acquisition device; and

an information presentation device,

wherein the information presentation device includes:

a first processor;

an image display unit; and

an operation input unit receiving an operation by a user, wherein

the first processor is programmed with instructions that, when executed, cause the first processor at least to:

receive an audio signal and an image signal;

make the display unit to display an image indicated by the image signal;

estimate direction information for each sound source based on the audio signal;

separate the audio signal to sound-source-classified audio signals for each sound source;

generate coordinate designation information indicating a part of a region of the image based on the operation by the user which is input to the operation input unit; and

select a sound-source-classified audio signal of a sound source associated with a coordinate which is included in a region indicated by the coordinate designation information, and which corresponds to the direction information.

6. The information transmission system according to claim 5 ,

wherein the information acquisition device is included in a robot, and includes:

a second processor; and

an imaging unit capturing an image and outputting an image signal of the image, and

the second processor is programmed with instructions that, when executed, cause the second processor at least to:

acquire position information of the imaging unit;

correct the image signal based on the position information, and

output the image signal to the information presentation device.

7. The image transmission system according to claim 5 ,

wherein the information acquisition device is included in a robot, and includes:

a second processor; and

an imaging unit capturing an image and outputting an image signal of the image,

the first processor of the information presentation device is programmed with instructions that, when executed, cause the first processor at least to:

generate, when a coordinate according to the direction information of the sound source corresponding to the sound-source-classified audio signal is beyond a display range of the image indicated by the input image signal, display information indicating that the coordinate is beyond the display range; and

generate control information which designates a conversion of an aiming of the imaging unit into a direction of the sound source corresponding to the sound-source classified audio signal, and

the second processor of the information acquisition device is

programmed with instructions that, when executed, cause the second processor at least to:

control the aiming of the imaging unit based on the control information; and

output the image signal to the information acquisition device.

8. An information presentation method of an information presentation device comprising:

a step of inputting an audio signal;

a step of inputting an image signal;

a step of displaying an image indicated by the image signal;

a step of estimating direction information for each sound source based on the audio signal;

a step of separating the audio signal to sound-source-classified audio signals for respective sound sources;

a step of receiving an operation input by a user and generating coordinate designation information indicating a part of a region of the image based on the operation by the user; and

a step of selecting sound-source-classified audio signals of sound sources according to a coordinate included in a region indicated by the coordinate designation information, that is, a coordinate corresponding to the direction information.

9. An information presentation method of an information presentation device comprising:

a step of inputting an image signal;

a step of inputting direction information for each sound source;

a step of receiving sound-source-classified audio signal for each sound source;

a step of receiving an operation input by a user and generating coordinate designation information indicating a part of a region of the image based on the operation by the user; and

a step of selecting sound-source-classified audio signals of sound sources according to a coordinate included in a region indicated by the coordinate designation information, that is, a coordinate corresponding to the direction information.

10. A non-transitory computer-readable storage medium comprising an information presentation program which causes a computer of an information presentation device to execute a process, the process comprising:

inputting an audio signal;

inputting an image signal;

displaying an image indicated by the image signal;

estimating direction information for each sound source based on the audio signal;

separating the audio signal to sound-source-classified audio signals for respective sound source;

receiving an operation input by a user and generating coordinate designation information indicating a part of a region of the image based on the operation by the user; and

selecting sound-source-classified audio signals of sound sources according to a coordinate included in a region indicated by the coordinate designation information, that is, a coordinate corresponding to the direction information.

11. A non-transitory computer-readable storage medium comprising an information presentation program which causes a computer of an information presentation device to execute a process, the process comprising:

inputting an image signal;

inputting direction information for each sound source;

receiving sound-source-classified audio signal for each sound source;

receiving an operation input by a user and generating coordinate designation information indicating a part of a region of the image based on the operation by the user; and

selecting sound-source-classified audio signals of sound sources according to a coordinate included in a region indicated by the coordinate designation information, that is, a coordinate corresponding to the direction information.