Dynamic interpolation for hybrid language models
In order to improve the accuracy of ASR, an utterance is transcribed using a plurality of language models, such as for example, an N-gram language model and a neural language model. The language models are trained separately. They each output a probability score or other figure of merit for a partial transcription hypothesis. Model scores are interpolated to determine a hybrid score. While recognizing an utterance, interpolation weights are chosen or updated dynamically, in the specific context of processing. The weights are based on dynamic variables associated with the utterance, the partial transcription hypothesis, or other aspects of context.
1. A method of speech transcription, the method comprising:
receiving a speech utterance;
inferring a sequence of phonemes from the speech utterance according to an acoustic model;
computing a plurality of transcription hypotheses from the sequence of phonemes;
computing, according to a first model, a first model score for the transcription;
computing, according to a second model, a second model score for the transcription;
computing a hybrid score by interpolation between the first model score and second model score using interpolation weights, where a first interpolation weight is applied to the first model score and a second interpolation weight is applied to the second model score and at least one of the weights is dynamic and conditioned on the content of the transcription based on word syntax structure and semantic information;
choosing one of the transcriptions based on the hybrid score; and
outputting the chosen hypothesis as the transcription of the speech.
2. The method of claim 1 , wherein the conditioning is based on word presence.
3. The method of claim 1 , wherein the conditioning is based on semantic information.
4. The method of claim 1 , wherein the first model is an n-gram model and the second model is a neural network.
5. The method of claim 1 , wherein the interpolation weights are generated using rule-based logic.
6. The method of claim 1 , wherein the interpolation weights are generated using a neural network.
7. The method of claim 1 , wherein the first model score and second model score are generated using a function that compresses the values of the first model score and the second model score.
8. The method of claim 1 , wherein the interpolation generates the hybrid score using a weighted sum function.
9. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for speech transcription, the method comprising:
receiving a speech utterance;
computing a plurality of transcription hypotheses from a sequence of phonemes inferred from an acoustic model;
computing, according to a first model, a first model score for the transcription;
computing, according to a second model, a second model score for the transcription;
computing a hybrid score by interpolation between the first model score and second model score using interpolation weights, where a first interpolation weight is applied to the first model score and a second interpolation weight is applied to the second model score and at least one of the weights is dynamic and conditioned on the content of the transcription based on word syntax structure and semantic information;
choosing one of the transcriptions based on the hybrid score; and
outputting the chosen hypothesis as the transcription of the speech.
10. The non-transitory computer readable storage medium of claim 9 , wherein the conditioning is based on word presence.
11. The non-transitory computer readable storage medium of claim 9 , wherein the conditioning is based on semantic information.
12. The non-transitory computer readable storage medium of claim 9 , wherein the first model is an n-gram model and the second model is a neural network.
13. The non-transitory computer readable storage medium of claim 9 , wherein the interpolation weights are generated using rule-based logic.
14. The non-transitory computer readable storage medium of claim 9 , wherein the interpolation weights are generated using a neural network.
15. The non-transitory computer readable storage medium of claim 9 , wherein the first model score and second model score are generated using a function that compresses the values of the first model score and the second model score.
16. The non-transitory computer readable storage medium of claim 9 , wherein the interpolation generates the hybrid score using a weighted sum function.