Method and apparatus for sound event detection robust to frequency change
View Patent ↗Disclosed is a sound event detecting method including receiving an audio signal, transforming the audio signal into a two-dimensional (2D) signal, extracting a feature map by training a convolutional neural network (CNN) using the 2D signal, pooling the feature map based on a frequency, and determining whether a sound event occurs with respect to each of at least one time interval based on a result of the pooling.
1. A sound event detecting method performed by a sound event detecting apparatus, the sound event detecting method comprising:
receiving an audio signal;
transforming the audio signal into a two-dimensional (2D) time-frequency signal;
extracting a feature map from the 2D signal using a trained convolutional neural network (CNN);
pooling the feature map based on a frequency; and
determining whether a sound event occurs with respect to each of one or more time intervals based on a result of the pooling.
2. The sound event detecting method of claim 1 , wherein the determining comprises:
calculating a probability value of a sound event occurring with respect to each of the one or more time intervals based on the result of the pooling; and
determining whether a sound event occurs with respect to each of the one or more time intervals based on the probability value.
3. The sound event detecting method of claim 2 , wherein the determining of whether a sound event occurs with respect to each of the one or more time intervals based on the probability value comprises determining that a sound event occurs at a time interval if a probability value corresponding to the time interval is greater than or equal to a predetermined value.
4. The sound event detecting method of claim 1 , further comprising:
classifying a sound event occurring at each time interval based on predefined sound event information.
5. The sound event detecting method of claim 1 , wherein the audio signal is transformed into the 2D signal using one of fast Fourier transform (FFT), constant Q transform (CQT), and Wavelet.
6. A non-transitory computer-readable medium storing instructions that when executed by one or more processors, cause the one or more processors to perform the method of claim 1 .
7. A sound event detecting apparatus, comprising:
a memory configured to store a control program;
one or more processors configured to operate based on the control program; and
a receiver configured to receive an audio signal from an outside,
wherein the control program is configured to perform:
receiving an audio signal from an outside,
transforming the audio signal into a two-dimensional (2D) time-frequency signal,
extracting a feature map from the 2D signal using a trained neural network (CNN),
pooling the feature map based on a frequency, and
determining whether a sound event occurs with respect to each of one or more time intervals based on a result of the pooling.
8. The sound event detecting apparatus of claim 7 , wherein the determining comprises:
calculating a probability value of a sound event occurring with respect to each of the one or more time intervals based on the result of the pooling; and
determining whether a sound event occurs with respect to each of the one or more time intervals based on the probability value.
9. The sound event detecting apparatus of claim 8 , wherein the determining of whether a sound event occurs with respect to each of the one or more time intervals based on the probability value comprises determining that a sound event occurs at a time interval if a probability value corresponding to the time interval is greater than or equal to a predetermined value.
10. The sound event detecting apparatus of claim 7 , wherein the control program is further configured to perform classifying a sound event occurring at each time, interval based on predefined sound event information.
11. The sound event detecting apparatus of claim 7 , wherein the audio signal is transformed into the 2D signal using one of fast Fourier transform (FFT), constant transform (CQT), and Wavelet.