Multilingual speech recognition system using text derived recognition models
View Patent ↗There is provided a novel approach for generating multilingual text-to-phoneme mappings for use in multilingual speech recognition systems. The multilingual mappings are based on the weighted output from a neural network text-to-phoneme model, trained on data mixed from several languages. The multilingual mappings used together with a branched grammar decoding scheme is able to capture both inter- and intra-language pronunciation variations which is ideal for multilingual speaker independent recognition systems. A significant improvement in overall system performance is obtained for a multilingual speaker independent name dialing task when applying multilingual instead of language dependent text-to-phoneme mapping.
1. A method of speech recognition in order to identify a speech command as a match to a written text command comprising the steps:
providing a text input from a text database;
receiving an acoustic input;
generating sequences of multilingual phoneme symbols based on said text input by means of a multilingual text-to-phoneme module;
generating variations of pronunciations which are recognizable in response to said sequences of multilingual phoneme symbols determined by use of a branched grammar; and
comparing said variations of pronunciations with the acoustic input in order to find a match.
2. A method according to claim 1 wherein the text input is processed letter by letter, and wherein a neural network provides an estimate of the posterior probabilities of the different phonemes for each letter.
3. A method according to claim 1 comprising deriving said text input from a database containing user entered text strings.
4. A method according to claim 1 , wherein generating variations of pronunciations determined by use of a branched grammar comprises use of a weighted branched grammar in which the weightings are representative of probabilities of the phonemes of said sequences of multilingual phoneme symbols.
5. A method according to claim 1 , wherein said sequences of multilingual phoneme symbols comprise a complete and non-redundant set of multilingual phoneme symbol sequences for languages supported by said multilingual text-to-phoneme module.
6. A method according to claim 1 , wherein generating sequences of multilingual phoneme symbols comprises generating a weighted branched grammar in which the weightings are representative of probabilities of the phonemes of said sequences of multilingual phoneme symbols.
7. A method according to claim 1 , wherein generating variations of pronunciations determined by use of a branched grammar comprises capturing intra-language and inter-language pronunciation variations of said text input in said branched grammar of said sequences of multilingual phoneme symbols.
8. A system for speech recognition comprising:
a text database for providing a text input;
transducer means for receiving an acoustic input;
a multilingual text-to-phoneme module for outputting sequences of multilingual phoneme symbols based on said text input;
a pronunciation lexicon module receiving said sequences of multilingual phoneme symbols from said multilingual text-to-phoneme module, and for generating variations of pronunciations which are recognizable in response thereto which are determined by a branched grammar; and
a multilingual recognizer based on multilingual acoustic phoneme models for comparing said variations of pronunciations generated by the pronunciation lexicon module with the acoustic input in order to find a match.
9. A system according to claim 8 , wherein the multilingual text-to phoneme module processes said text input letter by letter, and comprises a neural network for giving an estimate of the posterior probabilities of the different phonemes for each letter.
10. A system according to claim 9 wherein the neural network is a standard fully connected feed-forward multi-layer perceptron neural network.
11. A system according to claim 8 wherein the text input is derived from a database containing user entered text strings.
12. A system according to claim 11 wherein the database containing user entered text strings is an electronic phonebook including phone numbers and associated name labels.
13. A system according to claim 8 , wherein said pronunciation lexicon module uses a weighted branched grammar for determining variations of pronunciations in which the weightings are representative of probabilities of the phonemes of said sequences of multilingual phoneme symbols.
14. A system according to claim 8 , wherein said multilingual text-to-phoneme module generates said sequences of multilingual phoneme symbols to correspond to a complete and non-redundant set of multilingual phoneme symbol sequences for languages supported by said multilingual text-to-phoneme module.
15. A system according to claim 8 , wherein said multilingual text-to-phoneme module outputs the sequences of multilingual phoneme symbols in a weighted branched grammar in which the weightings are representative of probabilities of the phonemes of said sequences of multilingual phoneme symbols.
16. A system according to claim 8 , wherein said pronunciation lexicon module captures intra-language and inter-language pronunciation variations of said text input in said branched grammar of said sequences of multilingual phoneme symbols.
17. A communication terminal including a speech recognition unit comprising:
a text database for providing a text input;
transducer means for receiving an acoustic input;
a multilingual text-to-phoneme module for outputting sequences of multilingual phoneme symbols based on said text input;
a pronunciation lexicon module receiving said sequences of multilingual phoneme symbols from said multilingual text-to phoneme module, and for generating variations of pronunciations in response thereto which are determined by a branched grammar; and
a multilingual recognizer based on multilingual acoustic phoneme models for comparing said variations of pronunciations generated by the pronunciation lexicon module with the acoustic input in order to find a match.
18. A communication terminal according to claim 17 , wherein the multilingual text-to phoneme module processes said text input letter by letter, and comprises a neural network for giving an estimate of the posterior probabilities of the different phonemes for each letter.
19. A communication terminal according to claim 18 wherein the neural network is a standard fully connected feed-forward multi-layer perceptron neural network.
20. A communication terminal according to claim 17 wherein the text input is derived from a database containing user entered text strings.
21. A communication terminal according to claim 20 wherein the database containing user entered text strings is an electronic phonebook including phone numbers and associated name labels.
22. A communication terminal according to claim 17 , wherein said pronunciation lexicon module uses a weighted branched grammar for determining variations of pronunciations in which the weightings are representative of probabilities of the phonemes of said sequences of multilingual phoneme symbols.
23. A communication terminal according to claim 17 , wherein said multilingual text-to-phoneme module generates said sequences of multilingual phoneme symbols to correspond to a complete and non-redundant set of multilingual phoneme symbol sequences for languages supported by said multilingual text-to-phoneme module.
24. A communication terminal according to claim 17 , wherein said multilingual text-to-phoneme module outputs the sequences of multilingual phoneme symbols in a weighted branched grammar in which the weightings are representative of probabilities of the phonemes of said sequences of multilingual phoneme symbols.
25. A communication terminal according to claim 17 , wherein said pronunciation lexicon module captures intra-language and inter-language pronunciation variations of said text input in said branched grammar of said sequences of multilingual phoneme symbols.