HMM-based bilingual (Mandarin-English) TTS techniques

1. A method for generating speech based on text in one or more languages, implemented at least in part by a computer, the method comprising:

providing a phone set for a plurality of languages, the phone set comprising a union of phones of the plurality of languages;

training, for the plurality of languages, a multilingual hidden Markov model (HMM) comprising state level sharing across the plurality of languages based on language sentences in each of the plurality of languages without any sentences including a mixture of more than one language;

tying states of the multilingual HMM across the plurality of languages and clustering the tied states across the plurality of languages into a single decision based at least in part on a language independent question and a language specific question;

receiving text in one or more of the plurality of languages of the multilingual HMM; and

generating speech, for the received text, based at least in part on the multilingual HMM.

2. The method of claim 1 wherein the plurality of languages comprise English and/or Mandarin.

3. The method of claim 1 , wherein the tied states comprise context-dependent states.

4. A method for generating speech based on text, implemented at least in part by a computer, the method comprising:

building a first language specific decision tree;

building a second language specific decision tree;

mapping a leaf node from the first tree to a leaf node of the second tree using a Kullback-Leibler divergence (KLD) technique based on a spectral feature located in a subset of less than all of a frequency range for measuring the KLD between two hidden Markov models (HMMs);

receiving text in the second language; and

generating speech in the second language, for the received text, based at least in part on the mapping the leaf node from the first tree to the leaf node of the second tree.

5. The method of claim 4 further comprising mapping a leaf node from the second tree to a leaf node of the first tree.

6. The method of claim 4 wherein multiple leaf nodes of one decision tree map to a single leaf node of another decision tree.

7. The method of claim 4 wherein the first language comprises Mandarin.

8. The method of claim 4 wherein the first and the second language comprise English and Mandarin.

9. The method of claim 4 wherein the generating speech occurs without using speech provided in the second language.

10. A method for a multilingual text-to-speech (TTS) system, implemented at least in part by a computer, the method comprising:

providing a hidden Markov model (HMM) for a sound in a first language;

providing a HMM for a sound in a second language;

determining line spectral pairs for the sound in the first language;

determining line spectral pairs for the sound in the second language;

calculating a Kullback-Leibler divergence (KLD) score based at least on the line spectral pairs for the sound in the first language and the sound in the second language, wherein the KLD score indicates similarity/dissimilarity between the sound in the first language and the sound in the second language based on line spectral pairs that are independent of at least a line spectral pair located in an upper half of a frequency range used for measuring a Kullback-Leibler divergence; and

building a multilingual HMM-based TTS system wherein the TTS system comprises shared sounds based on KLD scores.

11. The method of claim 10 wherein the sound in the first language comprises a phone and wherein the sound in the second language comprises a phone.

12. The method of claim 10 wherein the sound in the first language comprises a sub-phone and wherein the sound in the second language comprises a sub-phone.

13. The method of claim 10 wherein the sound in the first language comprises a complex phone and wherein the sound in the second language comprises two or more phones.

14. The method of claim 10 wherein the sound in the first language comprises a context-dependent sound.