Electronic apparatus generating a sound of a plurality of channels and controlling method thereof
An electronic apparatus and/or a controlling method are provided. The electronic apparatus may include a camera for photographing an image, a microphone for receiving an input of a sound of a first channel, and a processor for generating sounds of a plurality of channels based on the input sound, wherein the processor is configured to identify an object and the location of the object from the photographed image, classify the input sound based on an audio source, and allot the sound to the corresponding identified object, copy the classified sound and generate sounds of two channels, adjust characteristics of the generated sounds of two channels based on the audio source allotted to the identified object and the location of the identified object, and mix the sounds of two channels wherein the characteristics were adjusted according to the audio source and generate a stereo sound of two channels.
1. An electronic apparatus comprising:
a camera configured to capture an image;
a display:
an input interface comprising circuitry;
a microphone configured to receive an input of a sound of a first channel;
a Time of Flight (ToF) sensor; and
a processor, comprising processing circuitry, configured to cause generation of sounds of a plurality of channels based on the input sound and to:
identify an object and the location of the object from the image by the ToF sensor,
classify the input sound based on an audio source, and allot the classified sound to the corresponding identified object,
copy the classified sound and control to generate sounds of at least two channels,
control to adjust characteristics of the generated sounds of at least two channels based on the audio source allotted to the identified object and the location of the identified object using at least one of sound panning to a predetermined location, time delay, phase delay, strength adjustment, amplitude adjustment and spectral modification, the location of the identified object being sensed by the ToF sensor,
control to mix the sounds of at least two channels wherein the characteristics were adjusted according to the audio source and generate a stereo sound of at least two channels,
based on failing to identify an object corresponding to the classified sound, control the display to display a predetermined indicator and the identified object on the classified sound, and
match the classified sound on which the predetermined indicator is displayed to the identified object based on a user's command input through the input interface.
2. The electronic apparatus of claim 1 ,
wherein the processor is configured to:
separate the input sound into respective sounds, identify audio sources corresponding to the respective separated sounds based on frequency characteristics, and classify the respective sounds based on the identified audio sources.
3. The electronic apparatus of claim 1 ,
wherein the processor is configured to:
identify the object and the location of the object based on an image processing artificial intelligence model, and adjust the characteristics of the generated sounds of two channels based on a sound processing artificial intelligence model.
4. An electronic apparatus comprising:
a camera configured to capture an image;
a display:
an input interface comprising circuitry:
a microphone configured to receive an input of a sound;
a Time of Flight (ToF) sensor and
a processor, comprising processing circuitry, configured to cause generation of sounds of a plurality of channels based on the input sound and to:
identify an object and the location of the object from the image,
classify the input sound based on an audio source, and allot the sound to the corresponding identified object,
extract a base sound based on the input sound, assume a rear sound, and cluster the allotted sound based on the location of the identified object,
control to adjust characteristics of the extracted base sound, the assumed rear sound, and the clustered sound using at least one of sound panning to a predetermined location, time delay, phase delay, strength adjustment, amplitude adjustment and spectral modification, the location of the identified object being sensed by the ToF sensor,
allot the sounds wherein the characteristics were adjusted to respective channels according to the audio sources and generate a surround sound,
based on failing to identify an object corresponding to the classified sound, control the display to display a predetermined indicator and the identified object on the classified sound, and
match the classified sound on which the predetermined indicator is displayed to the identified object based on a user's command input through the input interface.
5. The electronic apparatus of claim 4 ,
wherein the processor is configured to:
assume a sound other than the sound allotted to the identified object in the image among the input sound as the rear sound.
6. The electronic apparatus of claim 4 ,
wherein the processor is configured to:
divide the image into predetermined areas, and cluster sounds allotted to objects included in the same area among the respective divided areas in the same group.
7. The electronic apparatus of claim 4 ,
wherein the processor is configured to:
perform low-pass filtering of the input sound and extract the base sound.
8. A controlling method of an electronic apparatus, the method comprising:
capturing an image, and receiving an input of a sound; and
generating sounds of a plurality of channels based on the input sound,
wherein the generating the sounds of a plurality of channels comprises:
identifying an object and the location of the object from the image by ToF sensor of the electronic apparatus;
classifying the input sound based on an audio source, and allotting the classified sound to the corresponding identified object;
copying the classified sound and generating sounds of at least two channels;
adjusting characteristics of the generated sounds of at least two channels based on the audio source allotted to the identified object and the location of the identified object by method(s) using at least one of sound panning to a predetermined location, time delay, phase delay, strength adjustment, amplitude adjustment and spectral modification, the location of the identified object being sensed by the ToF sensor; and
mixing the sounds of at least two channels wherein the characteristics were adjusted according to the audio source and generating a stereo sound of at least two channels,
wherein the generating the sounds of a plurality of channels comprises:
based on failing to identify an object corresponding to the classified sound, displaying a predetermined indicator and the identified object on the classified sound, and
matching the sound on which the predetermined indicator is displayed to the identified object according to a user's command input.
9. The controlling method of an electronic apparatus of claim 8 ,
wherein the generating the sounds of a plurality of channels comprises:
separating the input sound into respective sounds, identifying audio sources corresponding to the respective separated sounds based on frequency characteristics, and classifying the respective sounds based on the identified audio sources.