System and method to correct for packet loss in ASR systems
A system and method are presented for the correction of packet loss in audio in automatic speech recognition (ASR) systems. Packet loss correction, as presented herein, occurs at the recognition stage without modifying any of the acoustic models generated during training. The behavior of the ASR engine in the absence of packet loss is thus not altered. To accomplish this, the actual input signal may be rectified, the recognition scores may be normalized to account for signal errors, and a best-estimate method using information from previous frames and acoustic models may be used to replace the noisy signal.
1. A method to correct for packet loss in an audio signal in automatic speech recognition systems comprising rescoring of phoneme probability calculations for a section of the audio signal, comprising the steps of:
accumulating probabilities for a given word in a lexicon using the highest scoring tokens for every frame of the audio signal;
determining when the probabilities for the given word are statistically significant;
reporting matches when the probabilities are statistically significant;
normalizing values of probabilities that are not statistically significant by replacing values for section of the audio signal comprising packet loss; and
rescoring the probabilities for the given word.
2. The method of claim 1 wherein the normalizing comprises deleting values for sections of the audio signal comprising packet loss.
3. The method of claim 1 wherein replacing values for sections of the audio signal comprising packet loss includes replacing a noisy signal caused by packet loss by a best estimate selected from historical values obtained offline or selected from previous frames and acoustic models.
4. The method of claim 1 , wherein the reporting comprises omitting low confidence hits.
5. The method of claim 1 , wherein the probabilities are accumulated from historical values previously obtained for each phoneme from a corpus without packet loss.
6. The method of claim 1 , wherein the rescoring comprises tagging the matches for the given word whose confidence values through their scores have been affected by packet loss using tagged packet information.
7. The method of claim 6 , wherein the tagged packet information is determined by extracting mel frequency cepstral coefficient features, wherein the frames of audio are decomposed into overlapping frames.
8. The method of claim 7 , wherein the overlapping frames are 20 ms with an overlap factor.
9. The method of claim 8 , wherein the overlap factor is 50%.
10. An automatic speech recognition system that corrects for packet loss in an audio signal comprising rescoring of phoneme probability calculations for a section of the audio signal, comprising:
a processor; and
a memory in communication with the processor, the memory storing instructions that, when executed by the processor, causes the processor to rescore phoneme probability calculations for a section of the audio signal by:
accumulating probabilities for a given word in a lexicon using the highest scoring tokens for every frame of the audio signal;
determining when the probabilities for the given word are statistically significant;
reporting matches when the probabilities are statistically significant;
normalizing values of probabilities that are not statistically significant by replacing values for sections of the audio signal comprising packet loss; and
rescoring the probabilities for the given word.
11. The system of claim 10 wherein the normalizing comprises deleting values for sections of the audio signal comprising packet loss.
12. The system of claim 10 wherein replacing values for sections of the audio signal comprising packet loss includes replacing a noisy signal caused by packet loss by a best estimate selected from historical values obtained offline or selected from previous frames and acoustic models.
13. The system of claim 10 , wherein the reporting comprises omitting low confidence hits.
14. The system of claim 10 , wherein the probabilities are accumulated from historical values previously obtained for each phoneme from a corpus without packet loss.
15. The system of claim 10 , wherein the rescoring comprises tagging the matches for the given word whose confidence values through their scores have been affected by packet loss using tagged packet information.
16. The system of claim 15 , wherein the tagged packet information is determined by extracting mel frequency cepstral coefficient features, wherein the frames of audio are decomposed into overlapping frames.
17. The system of claim 16 , wherein the overlapping frames are 20 ms with an overlap factor.
18. The system of claim 17 , wherein the overlap factor is 50%.