Automatic detection and attenuation of speech-articulation noise events
Described is a method of performing automatic audio enhancement on an input audio signal including at least one speech-articulation noise event. The method comprises: segmenting the input audio signal into a number of audio frames; obtaining at least one feature parameter from the audio frames; and determining, based at least in part on the obtained feature parameter, a respective type of the speech-articulation noise event and a respective time-frequency range associated with the speech-articulation noise event within the input audio signal.
1 . A method of performing automatic audio enhancement on an input audio signal including at least one speech-articulation noise event, the method comprising:
segmenting the input audio signal into a number of audio frames;
obtaining at least one feature parameter from the audio frames, wherein obtaining the at least one feature parameter from the audio frames comprises, for each audio frame, obtaining at least one measure of kurtosis based on time-domain sample amplitudes of the audio frames;
determining, based at least in part on the obtained feature parameter, a respective type of the speech-articulation noise event and a respective time-frequency range associated with the speech-articulation noise event within the input audio signal, wherein determining, based on the obtained feature parameter, the respective type of the speech-articulation noise event and the respective range thereof in the input audio signal comprises:
comparing the obtained measure of kurtosis to a predefined kurtosis threshold; and
if the measure of kurtosis exceeds the predefined kurtosis threshold, determining that the audio frame comprises a mouth click event, and determining start and end boundaries of the mouth click event based on respective positions at which the measure of kurtosis rises above and falls below the predefined kurtosis threshold; and
attenuating the speech-articulation noise event in accordance with a determined type of the speech-articulation noise event and the respective time-frequency range associated with the speech-articulation noise event.
2 . The method according to claim 1 , wherein the determined range comprises at least one boundary of the determined speech-articulation noise event, in the time or spectral domain.
3 . The method according to claim 1 , wherein the speech-articulation noise event comprises at least one of: a mouth click event or a speech plosive event.
4 . The method according to claim 3 , wherein the speech-articulation noise event comprises one or more mouth click events; and wherein the one or more mouth click events comprise at least one of: a non-speech click event, a speech click event, or a lip smack event.
5 . The method according to claim 4 , wherein, after segmenting the input audio signal into a number of audio frames, the method further comprises:
classifying the audio frames as either speech frames or non-speech frames.
6 . The method according to claim 5 , wherein the segmentation is performed by using two different time window sizes, one of the two time window sizes being shorter than the other.
7 . The method according to claim 6 , wherein the shorter time window size is used for detecting speech click events in the speech frames and a longer time window size is used for detecting non-speech click events in the non-speech frames.
8 . The method according to claim 5 , wherein obtaining at least one feature parameter from the audio frames comprises:
for each speech frame, obtaining a respective approximation of residual without speech harmonic components and a respective first measure of kurtosis of sample amplitudes for the approximation of residual, and
wherein determining, based on the obtained feature parameter, a respective type of the speech-articulation noise event and a respective range thereof in the input audio signal comprises:
comparing the obtained first measure of kurtosis to a first predefined kurtosis threshold; and
if the first measure of kurtosis exceeds the first predefined kurtosis threshold, determining that the speech frame comprises a speech click event, and determining start and end boundaries of the speech click event based on respective positions at which the first measure of kurtosis rises above and falls below the first predefined kurtosis threshold.
9 . The method according to claim 8 , wherein the approximation of residual without speech harmonic components is a second-order waveform difference.
10 . The method according to claim 8 , further comprising:
obtaining a second measure of kurtosis from residual sample amplitudes of the speech frame;
wherein the type and range of the speech-articulation noise event are determined based on the second measure of kurtosis relative to the first measure of kurtosis.
11 . The method according to claim 8 , further comprising:
refining the determined range of the speech click event by:
locating a sample position with the largest second-order difference within the determined range of the speech click event; and
determining the refined range of the speech click event by applying a predefined speech click event duration around the located sample position.
12 . The method according to claim 8 , further comprising:
determining the range of the speech click event further based on a min/max change rate calculated from local minima and maxima in the speech frame.
13 . The method according to claim 8 , further comprising:
attenuating the determined one or more mouth click events based on respective spectral gains derived from spectral envelopes of the audio frames containing detected mouth click events and target envelopes calculated based on respective reference frames.
14 . The method according to claim 5 , wherein obtaining at least one feature parameter from the audio frames comprises:
for each non-speech frame, obtaining a respective third measure of kurtosis of time-domain sample amplitudes in the non-speech frame, and
wherein determining, based on the obtained feature parameter, a respective type of the speech-articulation noise event and a respective range thereof in the input audio signal comprises:
comparing the obtained third measure of kurtosis to a second predefined kurtosis threshold; and
if the third measure of kurtosis exceeds the second predefined kurtosis threshold, determining that the non-speech frame comprises a non-speech click event; and determining start and end boundaries of the non-speech click event based on respective positions at which the third measure of kurtosis rises above and falls below the second predefined kurtosis threshold.
15 . The method according to claim 14 , further comprising:
if two neighboring non-speech click events are within a predefined gap threshold, merging the two neighboring non-speech click events into a single speech click event.
16 . The method according to claim 14 , wherein
for a determined non-speech click event in a non-speech frame immediately preceding a speech frame:
calculating a high/low-band peak ratio as an amplitude ratio between the largest peak above a predefined frequency and the largest peak below the predefined frequency; and
if the calculated high/low-band peak ratio is above a predefined ratio threshold, determining the non-speech click event as a lip smack event.
17 . The method according to claim 16 , further comprising:
refining the determined range of the lip smack event based on the high/low-band peak ratio, a spectral slope and an energy envelope.
18 . The method according to claim 5 , further comprising:
determining the speech-articulation noise event further based on the center of gravity, COG, calculated for the speech frames in accordance with a further predefined threshold, for distinguishing mouth click events from speech transients.
19 . An apparatus comprising a processor and a memory coupled to the processor storing instructions, that when executed by the processor, cause the apparatus to carry out a method of performing automatic audio enhancement on an input audio signal including at least one speech-articulation noise event, the method comprising:
segmenting the input audio signal into a number of audio frames;
obtaining at least one feature parameter from the audio frames, wherein obtaining the at least one feature parameter from the audio frames comprises, for each audio frame, obtaining at least one measure of kurtosis based on time-domain sample amplitudes of the audio frames;
determining, based at least in part on the obtained feature parameter, a respective type of the speech-articulation noise event and a respective time-frequency range associated with the speech-articulation noise event within the input audio signal, wherein determining, based on the obtained feature parameter, the respective type of the speech-articulation noise event and the respective range thereof in the input audio signal comprises:
comparing the obtained measure of kurtosis to a predefined kurtosis threshold; and
if the measure of kurtosis exceeds the predefined kurtosis threshold, determining that the audio frame comprises a mouth click event, and determining start and end boundaries of the mouth click event based on respective positions at which the measure of kurtosis rises above and falls below the predefined kurtosis threshold; and
attenuating the speech-articulation noise event in accordance with a determined type of the speech-articulation noise event and the respective time-frequency range associated with the speech-articulation noise event.
20 . A non-transitory computer-readable storage medium storing one or more programs comprising instructions that, when executed by a processor, cause the processor to carry out a method of performing automatic audio enhancement on an input audio signal including at least one speech-articulation noise event, the method comprising:
segmenting the input audio signal into a number of audio frames;
obtaining at least one feature parameter from the audio frames, wherein obtaining the at least one feature parameter from the audio frames comprises, for each audio frame, obtaining at least one measure of kurtosis based on time-domain sample amplitudes of the audio frames;
determining, based at least in part on the obtained feature parameter, a respective type of the speech-articulation noise event and a respective time-frequency range associated with the speech-articulation noise event within the input audio signal, wherein determining, based on the obtained feature parameter, the respective type of the speech-articulation noise event and the respective range thereof in the input audio signal comprises:
comparing the obtained measure of kurtosis to a predefined kurtosis threshold; and
if the measure of kurtosis exceeds the predefined kurtosis threshold, determining that the audio frame comprises a mouth click event, and determining start and end boundaries of the mouth click event based on respective positions at which the measure of kurtosis rises above and falls below the predefined kurtosis threshold; and
attenuating the speech-articulation noise event in accordance with a determined type of the speech-articulation noise event and the respective time-frequency range associated with the speech-articulation noise event.