Method and apparatus for recognition of sound events based on convolutional neural network
Provided is a sound event recognition method that may improve a sound event recognition performance using a correlation between difference sound signal feature parameters based on a neural network, in detail, that may extract a sound signal feature parameter from a sound signal including a sound event, and recognize the sound event included in the sound signal by applying a convolutional neural network (CNN) trained using the sound signal feature parameter.
1. A method for recognition of sound events based on a convolutional neural network (CNN), the method comprising:
identifying a sound signal including a sound event;
extracting plurality of sound signal feature parameters from the sound signal; and
recognizing the sound event included in the sound signal by applying a CNN trained using the extracted plurality of sound signal feature parameters,
wherein the CNN is trained through a first hidden layer including at least one feature map generated by normalizing the extracted plurality of sound signal feature parameters, and
wherein the number of components of a remaining sound signal feature parameter is adjusted based on the number of components of a sound signal feature parameter preset for a frame interval to which a windowing is applied of sound signal.
2. The method of claim 1 , wherein the plurality of sound signal feature parameters are a scalogram which-divides the identified sound signal into preset frame units, and applies a discrete wavelet transform (DWT) to the frame units.
3. The method of claim 1 , wherein the plurality of sound signal feature parameters are a log spectrogram which divides the identified sound signal into preset frame units, and applies a log to a result of discrete Fourier transform (DFT) after applying windowing to the frame units.
4. The method of claim 3 , wherein the log spectrogram has a trade-off relationship between a frequency resolution and a time resolution based on a length of the windowing.
5. The method of claim 1 , wherein the plurality of sound signal feature parameters are a log Mel-filter bank energy which divides the identified sound signal into preset frame units, and applies a Mel-filter bank and a log to a result of DFT after applying windowing to the frame units, and
the Mel-filter bank applies a Mel scale to the result of DFT in view of a different resolution depending on a frequency.
6. The method of claim 5 , wherein the plurality of sound signal feature parameters are a cepstrogram is derived using a discrete cosine transform (DCT) coefficient derived by applying DCT to the log Mel-filter bank energy, a delta coefficient, and a delta-delta coefficient.
7. A method of training a convolutional neural network (CNN) to be applied to a sound event recognition method, the method comprising:
normalizing a plurality of sound signal feature parameters extracted from a sound signal including a sound event; and
training a CNN through a first hidden layer including at least one feature map generated based on the normalized sound signal feature parameter,
wherein the number of components of a remaining sound signal feature parameter is adjusted based on the number of components of a sound signal feature parameter preset for a frame interval to which a windowing is applied of sound signal.
8. The method of claim 7 , wherein the plurality of sound signal feature parameters includes
i) a scalogram which divides the identified sound signal into preset frame units, and applies a discrete wavelet transform (DWT) to the frame units,
ii) a log spectrogram which divides the identified sound signal into preset frame units, and applies a log to a result of discrete Fourier transform (DFT) after applying windowing to the frame units,
iii) a log Mel-filter bank energy which divides the identified sound signal into preset frame units, and applies a Mel-filter bank and a log to a result of DFT after applying windowing to the frame units, and
iv) a cepstrogram derived using a discrete cosine transform (DCT) coefficient derived by applying DCT to the log Mel-filter bank energy, a delta coefficient, and a delta-delta coefficient.
9. A sound event recognition apparatus to which a convolutional neural network (CNN) is applied, the sound event recognition apparatus comprising:
a processor and a memory including computer-readable instructions,
wherein, when the instructions are executed by the processor, the processor is configured to identify a sound signal including a sound event, extract a plurality of sound signal feature parameters from the sound signal, and recognize the sound event included in the sound signal by applying a CNN trained using the extracted plurality of sound signal feature parameters,
wherein the CNN is trained through a first hidden layer including at least one feature map generated by normalizing the extracted plurality of sound signal feature parameters, and
wherein the number of components of a remaining sound signal feature parameter is adjusted based on the number of components of a sound signal feature parameter preset for a frame interval to which the windowing is applied.
10. The sound event recognition apparatus of claim 9 , wherein the plurality of sound signal feature parameters are a scalogram which divides the identified sound signal into preset frame units, and applies a discrete wavelet transform (DWT) to the frame units.
11. The sound event recognition apparatus of claim 9 , wherein the plurality of sound signal feature parameters are a log spectrogram which divides the identified sound signal into preset frame units, and applies a log to a result of discrete Fourier transform (DFT) after applying windowing to the frame units.
12. The sound event recognition apparatus of claim 11 , wherein the log spectrogram has a trade-off relationship between a frequency resolution and a time resolution based on a length of the windowing.
13. The sound event recognition apparatus of claim 9 , wherein the plurality of sound signal feature parameters are a log Mel-filter bank energy which divides the identified sound signal into preset frame units, and applies a Mel-filter bank and a log to a result of DFT after applying windowing to the frame units, and
the Mel-filter bank applies a Mel scale to the result of DFT in view of a different resolution depending on a frequency.
14. The sound event recognition apparatus of claim 13 , wherein the plurality of sound signal feature parameters are a cepstrogram derived using a discrete cosine transform (DCT) coefficient derived by applying DCT to the log Mel-filter bank energy, a delta coefficient, and a delta-delta coefficient.