Electronic device for recognizing sound and method thereof
An example sound recognition method may include sampling input sound based on a preset sampling rate; performing Fast Fourier Transform (FFT) on the sampled input sound based on at least one of random FFT numbers or random hop lengths, and generating a two-dimensional (2D) feature map with a time axis and a frequency axis from the sampled input sound on which FFT is performed; training a neural network model, which recognizes sound, with a plurality of 2D feature maps including the first 2D feature map and an n th 2D feature map as training data.
1. A sound recognition method comprising:
sampling input sound based on a preset sampling rate; and
performing Fast Fourier Transform (FFT) on the sampled input sound based on at least one of random FFT numbers or random hop lengths, and generating a two-dimensional (2D) feature map, with a time axis and a frequency axis, from the sampled input sound on which FFT is performed,
wherein the generating of the 2D feature map comprises:
transforming the sampled input sound into first FFT data based on at least one of a first FFT number among the random FFT numbers or a first hop length among the random hop lengths, generating a first 2D feature map including a first feature from the first FFT data, transforming the sampled input sound into n th FFT data based on at least one of an n th FFT number among the random FFT numbers and an n th hop length among the random hop lengths, and generating an n th 2D feature map including an n th feature from the n th FFT data, where n is greater than 1; and
training a neural network model, which recognizes sound, with a plurality of 2D feature maps including the first 2D feature map and the n th 2D feature map as training data.
2. The sound recognition method as claimed in claim 1 , wherein the generating of the 2D feature map further comprises performing FFT on a sound part of a first time corresponding to a preset number of samples to generate a first partial feature map, performing FFT on a sound part of a second time consecutive to the first time and corresponding to the preset number of samples to generate a second partial feature map, performing FFT on a sound part of an n th time consecutive to an (n th −1) time and corresponding to the preset number of samples to generate an n th partial feature map, and connecting the first partial feature map, the second partial feature map, and the n th partial feature map to one another.
3. The sound recognition method as claimed in claim 1 , wherein the generating of the 2D feature map further comprises determining an amount of data on the frequency axis according to the random FFT numbers.
4. The sound recognition method as claimed in claim 1 , wherein the generating of the 2D feature map further comprises changing a resolution of the frequency axis of the 2D feature map based on the random FFT numbers.
5. The sound recognition method as claimed in claim 1 , wherein the generating of the 2D feature map further comprises generating a plurality of 2D feature maps including different features from each other by hopping the number of samples corresponding to the random hop lengths m times, and
the training of the neural network model comprises training the neural network model using the plurality of 2D feature maps as a plurality of pieces of training data.
6. The sound recognition method as claimed in claim 5 , wherein the generating of the 2D feature map comprises changing a resolution of the time axis of the 2D feature map based on the random hop lengths.
7. The sound recognition method as claimed in claim 1 , further comprising masking a preset region of the 2D feature map.
8. The sound recognition method as claimed in claim 7 , wherein the masking of the preset region of the 2D feature map comprises masking a region of the generated 2D feature map in which an amount of valid sound data is greater than a preset first level at a preset first ratio, and masking a region of the generated 2D feature map in which an amount of valid sound data is equal to or less than a preset second level at a preset second ratio, wherein the preset second ratio is higher than the preset first ratio.
9. The sound recognition method as claimed in claim 7 , wherein the masked preset region comprises at least one of a region of a geometric shape that is long on the time axis, a region of a geometric shape that is long on the frequency axis, a diagonal region with a preset width, or a preset region with a grid pattern.
10. The sound recognition method as claimed in claim 7 , wherein the masking of the preset region of the 2D feature map comprises masking different regions of the plurality of 2D feature maps, and
the training of the neural network model comprises training a neural network model, which recognizes sound, with the plurality of 2D feature maps, in which the different regions are masked, as a plurality of pieces of training data.
11. The sound recognition method as claimed in claim 1 , further comprising:
receiving first sound to be recognized;
generating a 2D feature map to be recognized from the received first sound based on a preset FFT number and a preset hop length; and
recognizing the sound by inputting the generated 2D feature map to a trained neural network model.
12. The sound recognition method as claimed in claim 11 , wherein the recognizing of the sound comprises recognizing at least one of a type of the sound or a place in which the sound is generated.
13. The sound recognition method as claimed in claim 11 , further comprising:
identifying a first feature vector of the received first sound to be recognized;
identifying a second feature vector of second sound to be recognized when the second sound is input; and
comparing the first feature vector and the second feature vector with each other to identify whether the second sound is generated from the same source as the first sound.
14. An electronic device comprising:
an input interface; and
at least one processor,
wherein the at least one processor is configured to:
sample sound input through the input interface based on a preset sampling rate;
perform Fast Fourier Transform (FFT) on the sampled input sound based on at least one of random FFT numbers or random hop lengths, and generate a two-dimensional (2D) feature map with a time axis and a frequency axis from the sampled input sound on which FFT is performed;
transform the sampled input sound into first FFT data based on at least one of a first FFT number among the random FFT numbers or a first hop length among the random hop lengths, generate a first 2D feature map including a first feature from the first FFT data, transform the sampled input sound into n th FFT data based on at least one of an n th FFT number among the random FFT numbers and an n th hop length among the random hop lengths, and generate an n th 2D feature map including an n th feature from the n th FFT data, wherein n is greater than 1; and
train a neural network model, which recognizes sound, with a plurality of 2D feature maps including the first 2D feature map and the n th 2D feature map as training data.
15. The electronic device as claimed in claim 14 , wherein the at least one processor is further configured to perform FFT on a sound part of a first time corresponding to a preset number of samples to generate a first partial feature map, perform FFT on a sound part of a second time consecutive to the first time and corresponding to the preset number of samples to generate a second partial feature map, perform FFT on a sound part of an n th time consecutive to an (n th −1) time and corresponding to the preset number of samples to generate an n th partial feature map, and connect the first partial feature map, the second partial feature map, and the n th partial feature map to one another to generate the 2D feature map.