Voice activity detection using a soft decision mechanism
View Patent ↗Voice activity detection (VAD) is an enabling technology for a variety of speech based applications. Herein disclosed is a robust VAD algorithm that is also language independent. Rather than classifying short segments of the audio as either “speech” or “silence”, the VAD as disclosed herein employees a soft-decision mechanism. The VAD outputs a speech-presence probability, which is based on a variety of characteristics.
1. A method of detection of voice activity in audio data, the method comprising:
obtaining audio data;
segmenting the audio data into a plurality of frames;
calculating a plurality of features for each frame, wherein each of the plurality of features, comprises a different measurement of the energy of the audio data in the frame;
combining the plurality of features mathematically to form an activity probability for each frame, wherein the activity probability for each frame corresponds to the likelihood that the frame contains speech;
calculating, for each frame, a moving average of the activity probability, wherein the moving average for a particular frame is the average of the activity probabilities of group of consecutive frames including the particular frame;
selecting, for each frame, a threshold, wherein the selection for a particular frame depends on the threshold selected for a frame prior to the particular frame;
comparing, for each frame, the calculated moving average and the selected threshold;
based on the comparison for each frame either (i) marking the frame as a boundary between speech and non-speech or (ii) not marking the frame;
identifying speech and non-speech segments in the audio data based on the marked frames; and
deactivating subsequent processing of non-speech segments in the audio data to save computational bandwidth.
2. The method of detection of voice activity in audio data of claim 1 , wherein the calculating a plurality of features for each frame includes calculating an overall energy speech probability for each frame.
3. The method of detection of voice activity in audio data of claim 1 , wherein the calculating a plurality of features for each frame includes calculating a band energy speech probability for each frame.
4. The method of detection of voice activity in audio data of claim 1 , wherein the calculating a plurality of features for each frame includes calculating a spectral peakiness speech probability for each frame.
5. The method of detection of voice activity in audio data of claim 1 , wherein the calculating a plurality of features for each frame includes calculating a residual energy speech probability for each frame.
6. The method of detection of voice activity in audio data of claim 1 , wherein the obtaining step includes obtaining a set of audio data in segmented form.
7. A non-transitory computer readable medium having computer executable instructions for performing a method comprising:
obtaining audio data;
segmenting the audio data into a plurality of frames;
calculating a plurality of features for each frame, wherein each of the plurality of features, comprises a different measurement of the energy of the audio data in the frame;
combining the plurality of features mathematically to form an activity probability for each frame, wherein the activity probability for each frame corresponds to the likelihood that the frame contains speech;
calculating, for each frame, a moving average of the activity probability, wherein the moving average for a particular frame is the average of the activity probabilities of group of consecutive frames including the particular frame;
selecting, for each frame, a threshold, wherein the selection for a particular frame depends on the threshold selected for a frame prior to the particular frame;
comparing, for each frame, the calculated moving average and the selected threshold;
based on the comparison for each frame either (i) marking the frame as a boundary between speech and non-speech or (ii) not marking the frame;
identifying speech and non-speech segments in the audio data based on the marked frames; and
deactivating subsequent processing of non-speech segments in the audio data to save computational bandwidth.
8. The non-transitory computer readable medium of claim 7 , wherein the calculating a plurality of features for each frame includes calculating an overall energy speech probability for each frame.
9. The non-transitory computer readable medium of claim 7 , wherein the calculating a plurality of features for each frame includes calculating a band energy speech probability for each frame.
10. The non-transitory computer readable medium of claim 7 , wherein the calculating a plurality of features for each frame includes calculating a spectral peakiness speech probability for each frame.
11. The non-transitory computer readable medium of claim 7 , wherein the calculating a plurality of features for each frame includes calculating a residual energy speech probability for each frame.
12. The non-transitory computer readable medium of claim 7 , wherein the obtaining step includes obtaining a set of audio data in segmented form.
13. A method of detection of voice activity in audio data, the method comprising:
obtaining audio data;
segmenting the audio data into a plurality of frames;
calculating a probability corresponding to the overall energy of the audio data in each of the plurality of frames;
calculating a probability corresponding to the band energy of the audio data in each of the plurality of frames;
calculating a probability corresponding to the spectral peakiness of the audio data in each of the plurality of frames;
calculating a probability corresponding to the residual energy of the audio data in each of the plurality of frames;
computing an activity probability for each of the plurality of frames from the probabilities corresponding to the overall energy, band energy, spectral peakiness, and residual energy;
calculating, for each of the plurality of frames, a moving average of the activity probability, wherein the moving average for a particular frame is the average of the activity probabilities of group of consecutive frames including the particular frame;
comparing the moving average of each frame to at least one threshold; and
based on the comparison for each frame either (i) marking the frame as a boundary between speech and non-speech or (ii) not marking the frame;
identifying speech and non-speech segments in the audio data based on the marked frames; and
deactivating subsequent processing of non-speech segments in the audio data to save computational bandwidth.