Token confidence scores for automatic speech recognition
Methods and systems for correction of a likely erroneous word in a speech transcription are disclosed. By evaluating token confidence scores of individual words or phrases, the automatic speech recognition system can replace a low-confidence score word with a substitute word or phrase. Among various approaches, neural network models can be used to generate individual confidence scores. Such word substitution can enable the speech recognition system to automatically detect and correct likely errors in transcription. Furthermore, the system can indicate the token confidence scores on a graphic user interface for labeling and dictionary enhancement.
1. A computer-implemented method for speech recognition, comprising:
receiving, at an automatic speech recognition system, an utterance;
generating, based on an acoustic model, a phoneme sequence of the utterance;
generating, based on the acoustic model, a plurality of phoneme sequences of the utterance that comprise the phoneme sequence;
assigning sentence-level acoustic scores to individual phoneme sequences to indicate the likelihood of correctness to represent the utterance;
segmenting the phoneme sequence into a token sequence that represents the phoneme sequence based on a pronunciation dictionary;
assigning respective token confidence scores to individual tokens in the token sequence, wherein a token confidence score represents a level of confidence of the correct representation of a tokenized word;
determining that a first confidence score associated with a token is lower than a predetermined threshold;
determining a substitute token associated with a second confidence score that is higher than the first confidence score; and
updating the token sequence by replacing the token with the substitute token.
2. The computer-implemented method of claim 1 , following updating the token sequence, further comprising:
reassigning the respective sentence-level acoustic scores to individual phoneme sequences; and
determining a phoneme hypothesis from the plurality of phoneme sequences to represent the utterance.
3. The computer-implemented method of claim 1 , wherein a confidence score model assigns the respective token confidence scores to individual tokens in the token sequence.
4. The computer-implemented method of claim 1 , wherein a translation model determines the substitute token and update the token sequence by replacing the token with the substitute token.
5. The computer-implemented method of claim 1 , wherein the token confidence scores are based on one or more of a token sequence probability analysis, acoustic probability analysis, and semantic analysis.
6. The computer-implemented method of claim 1 , further comprising:
generating, based on the phoneme sequence, a text transcription of the utterance.
7. A computer-implemented method for speech recognition, comprising:
generating, based on an acoustic model, a plurality of phoneme sequences of an utterance;
assigning sentence-level acoustic scores to individual phoneme sequences to indicate the likelihood of correctness to represent the utterance;
receiving, at an acoustic model of an automatic speech recognition system, a plurality of tokens and phrases representing the utterance;
assigning, by a confidence score model, respective token confidence scores and phrase confidence scores to the plurality of tokens and phrases, wherein a token confidence score and phrase confidence score represent a level of confidence of the correct representation of a word or phrase;
determining, by a translation model, that a first confidence score associated with a phrase is lower than a predetermined threshold;
determining a substitute phrase associated with a second confidence score that is higher than the first confidence score; and
replacing the phrase with the substitute phrase to generate an updated phoneme sequence.
8. The computer-implemented method of claim 7 , wherein the translation model determines a substitute token and update the token sequence by replacing the token with the substitute token.
9. The computer-implemented method of claim 7 , further comprising:
receiving the utterance of a query sentence;
generating a phoneme sequence of the utterance; and
segmenting the phoneme sequence into a plurality of tokens and phrases based on a pronunciation dictionary, wherein a phrase comprises one or more tokens.
10. The computer-implemented method of claim 7 , following replacing the phrase with the substitute phrase, further comprising:
reassigning the respective sentence-level acoustic scores to individual phoneme sequences; and
determining a phoneme hypothesis from the plurality of phoneme sequences to represent the utterance.
11. A computer-implemented method for speech recognition, comprising:
receiving, at an automatic speech recognition system, an utterance;
generating, based on an acoustic model, a phoneme sequence of the utterance;
generating, based on the acoustic model, a plurality of phoneme sequences of the utterance that comprise the phoneme sequence;
assigning sentence-level acoustic scores to individual phoneme sequences to indicate the likelihood of correctness to represent the utterance;
segmenting the phoneme sequence into a token sequence that represents the phoneme sequence based on a pronunciation dictionary;
assigning respective token confidence scores to individual tokens in the token sequence, wherein a token confidence score represents a level of confidence of the correct representation of a tokenized word;
determining that a first confidence score associated with a token is lower than a predetermined threshold;
determining a substitute token associated with a second confidence score that is higher than the first confidence score;
updating the token sequence by replacing the token with the substitute token; and
determining a phoneme hypothesis from the plurality of phoneme sequences to represent the utterance.
12. The computer-implemented method of claim 11 , wherein a confidence score model assigns the respective token confidence scores to individual tokens in the token sequence.
13. The computer-implemented method of claim 11 , wherein a translation model determines the substitute token and update the token sequence by replacing the token with the substitute token.
14. The computer-implemented method of claim 11 , wherein the token confidence scores are based on one or more of a token sequence probability analysis, acoustic probability analysis, and semantic analysis.
15. The computer-implemented method of claim 11 , further comprising:
generating, based on the phoneme sequence, a text transcription of the utterance.