Generating acoustic models of alternative pronunciations for utterances spoken by a language learner in a non-native language
View Patent ↗A non-transitory processor-readable medium storing code representing instructions to be executed by a processor includes code to cause the processor to receive acoustic data representing an utterance spoken by a language learner in a non-native language in response to prompting the language learner to recite a word in the non-native language and receive a pronunciation lexicon of the word in the non-native language. The pronunciation lexicon includes at least one alternative pronunciation of the word based on a pronunciation lexicon of a native language of the language learner. The code causes the processor to generate an acoustic model of the at least one alternative pronunciation in the non-native language and identify a mispronunciation of the word in the utterance based on a comparison of the acoustic data with the acoustic model. The code causes the processor to send feedback related to the mispronunciation of the word to the language learner.
1. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to:
receive acoustic data representing an utterance spoken by a language learner in a non-native language in response to prompting the language learner to recite a word in the non-native language;
receive a pronunciation lexicon of the word in the non-native language, the pronunciation lexicon of the word including at least one alternative pronunciation of the word determined based on a pronunciation lexicon of a native language of the language learner, the at least one alternative pronunciation of the word being a phonological error in the non-native language and having a probability greater than a threshold level of being spoken by the language learner when the language learner attempts to recite the word in the non-native language;
generate an acoustic model of the at least one alternative pronunciation of the word from the pronunciation lexicon of the word in the non-native language;
identify a mispronunciation of the word in the utterance based on a comparison of the acoustic data with the at least one alternative pronunciation of the word that is included in the acoustic model; and
send feedback related to the mispronunciation of the word to the language learner, wherein
the acoustic data is first acoustic data, the code to cause the processor to generate the acoustic model includes code to cause the processor to:
generate a maximum likelihood native model based on native data, the native data including a corpus of audio data provided by at least one native speaker,
generate a mix-up native triphone model based on the maximum likelihood native model, and
generate a maximum likelihood non-native model based at least on the mix-up native triphone model and second acoustic data.
2. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to:
receive acoustic data representing an utterance spoken by a language learner in a non-native language in response to prompting the language learner to recite a word in the non-native language;
receive a pronunciation lexicon of the word in the non-native language, the pronunciation lexicon of the word including at least one alternative pronunciation of the word determined based on a pronunciation lexicon of a native language of the language learner, the at least one alternative pronunciation of the word being a phonological error in the non-native language and having a probability greater than a threshold level of being spoken by the language learner when the language learner attempts to recite the word in the non-native language;
generate an acoustic model of the at least one alternative pronunciation of the word from the pronunciation lexicon of the word in the non-native language;
identify a mispronunciation of the word in the utterance based on a comparison of the acoustic data with the at least one alternative pronunciation of the word that is included in the acoustic model; and
send feedback related to the mispronunciation of the word to the language learner, wherein
the acoustic data is first acoustic data, the code to cause the processor to generate the acoustic model includes code to cause the processor to:
generate a maximum likelihood non-native model based at least in part on second acoustic data and a mix-up native triphone model generated by a maximum likelihood native model training process, and
generate the acoustic model as a minimum phone error trained acoustic model based on the maximum likelihood non-native model.