Method and system for text-to-speech synthesis
There is disclosed a method of generating a text-to-speech (TTS) training set for training a Machine Learning Algorithm (MLA) for generating machine-spoken utterances The method is executable by a server. The method includes generating a synthetic word based on merging separate phonemes from each of two words of a corpus of pre-recorded utterances, the merging being done using the common phoneme as a merging anchor, the merging resulting in at least two synthetic words. The synthetic words and assessor labels are used to train a classifier to predict a quality parameter associated with a new synthetic phonemes-based word, the quality parameter being representative of whether the new synthetic phonemes-based word is naturally sounding (based on acoustic features of generated synthetic words utterances). The classifier is then used to generate training objects for the MLA and to use the MLA to process the corpus of pre-recorded utterances into their respective vectors.
1. A method of generating a text-to-speech (TTS) training set for training a Machine Learning Algorithm (MLA) for generating machine-spoken utterances representing an inputted text, the method executable by a server, the method comprising:
receiving a word-corpus, each word of the word-corpus having:
a textual representation, and
a spoken representation;
parsing the textual representation of each word of the word-corpus into at least one phoneme;
for a given two words from the word-corpus, the two words having a common phoneme in-between them:
generating two synthetic words based on merging phonemes from each of the two words, the merging being done using the common phoneme as a merging anchor, the merging resulting in the two synthetic words, the merging including:
merging a first phoneme from a first word of the given two words, the first phoneme preceding the merging anchor with a second phoneme from a second word of the given two words, the second phoneme being after the merging anchor; and
merging a third phoneme from the first word of the given two words, the third phoneme being after the merging anchor with a fourth phoneme from the second word of the given two words, the fourth phoneme preceding the merging anchor,
generating synthetic words utterances based on the generated two synthetic words,
receiving, from at least one human assessor, scores for the synthetic words utterances, the scores being representative of a human-appreciation parameter of a phoneme transition pattern between phonemes across the anchor, the human-appreciation parameter being indicative of the transition pattern being naturally sounding, and
determining acoustic features of the synthetic words utterances; and
training a classifier based on the synthetic words utterances, the acoustic features and the scores representative of the human-appreciation parameter of the phoneme transition pattern to generate:
a quality parameter associated with a new synthetic phonemes-based word, the quality parameter being representative of whether the new synthetic phonemes-based word is naturally sounding.
2. The method of claim 1 , the method further comprising:
generating the TTS training set for training the MLA, the TTS training set including a plurality of training objects, each of the plurality of training objects including:
a respective new synthetic phonemes-based word;
an associated quality parameter generated by the classifier.
3. The method of claim 2 , wherein the MLA is a Deep Structured Semantic Model (DSSM) based MLA.
4. The method of claim 3 , wherein the DSSM based MLA has two sub-networks, and wherein training the MLA comprises:
feeding a first sub-network: a plurality of left sides of the new synthetic phonemes-based word and associated quality parameters;
feeding a second sub-network a plurality of right sides of the new synthetic phonemes-based word and associated quality parameters;
training the DSSM based MLA to generate vectors such that vectors of a given left side and a given right side that are associated with a comparatively higher associated quality parameter are also associated with a comparatively higher scalar multiplication value.
5. The method of claim 4 , wherein each of the new synthetic phonemes-based word is processed as a potential left side and a potential right side.
6. The method of claim 4 , wherein the training the DSSM based MLA to generate vectors further comprises training the DSSM based MLA such that vectors of another given left side and another given right side that are associated with a comparatively lower associated quality parameter are also associated with a comparatively lower scalar multiplication value.
7. The method of claim 6 , the method further comprising:
processing an entirety of phonemes of the word-corpus through the first sub-network and the second sub-network to generate a respective left-side vector and the right-side vector of each of the entirety of phonemes;
storing a plurality of generated left-side vectors and right-side vectors in a memory storage.
8. The method of claim 7 , wherein processing the entirety of phonemes comprises executing an embedding algorithm to generate a vector based on the phoneme.
9. The method of claim 7 , wherein the plurality of generated left-side vectors and right-side vectors is used during an in-use-phase to generate a cost function of the machine-spoken utterances during processing the inputted text.
10. The method of claim 9 , wherein the method further comprises, during the in-use-phase, calculating the cost function by multiplication a first matrix by a second matrix, the first matrix comprising the left-side vectors and the second matrix comprising the right-side vectors.
11. The method of claim 1 , wherein the acoustic features comprises at least one of a fundamental frequency and mel-frequency cepstrum (MFC).
12. The method of claim 11 , wherein a given acoustic feature of a given phoneme is generated based on the given phoneme and its context.
13. The method of claim 12 , wherein the context of the given phoneme is at least one adjacent other phoneme.
14. The method of claim 1 , wherein the classifier is implemented as a decision tree based MLA.
15. The method of claim 1 , wherein each of the two synthetic words has only a single merging anchor.
16. A computing device for generating a text-to-speech (TTS) training set for training a Machine Learning Algorithm (MLA) for generating machine-spoken utterances representing an inputted text, the MLA executed by the computing device, the computing device including a processor, the processor configured to:
receive a word-corpus, each word of the word-corpus having:
a textual representation, and
a spoken representation;
parse the textual representation of each word of the word-corpus into at least one phoneme;
for a given two words from the word-corpus, the two words having a common phoneme in-between them:
generate two synthetic words based on merging phonemes from each of the two words, the merging being done using the common phoneme as a merging anchor, the merging resulting in the two synthetic words, the merging including:
merging a first phoneme from a first word of the given two words, the first phoneme preceding the merging anchor with a second phoneme from a second word of the given two words, the second phoneme being after the merging anchor; and
merging a third phoneme from the first word of the given two words, the third phoneme being after the merging anchor with a fourth phoneme from the second word of the given two words, the fourth phoneme preceding the merging anchor,
generate synthetic words utterances based on the generated at least two synthetic words,
receive, from at least one human assessor, scores for the synthetic words utterances, the scores being representative of a human-appreciation parameter of a phoneme transition pattern between phonemes across the anchor, the human-appreciation parameter being indicative of the transition pattern being naturally sounding, and
determine acoustic features of the synthetic words utterances; and
train a classifier based on the synthetic words utterances, the acoustic features and the scores representative of the human-appreciation parameter of the phoneme transition pattern to generate:
a quality parameter associated with a new synthetic phonemes-based word, the quality parameter being representative of whether the new synthetic phonemes-based word is naturally sounding.
17. The method of claim 1 , wherein merging the first phoneme with the second phoneme comprises concatenating the first phoneme, the common phoneme, and the second phoneme, and wherein merging the third phoneme with the fourth phoneme comprises concatenating the third phoneme, the common phoneme, and the fourth phoneme.
18. The method of claim 1 , wherein generating the synthetic word utterances based on the generated two synthetic words comprises:
concatenating a first utterance corresponding to the first phoneme, a second utterance corresponding to the common phoneme, and a third utterance corresponding to the second phoneme; and
concatenating a fourth utterance corresponding to the third phoneme, the second utterance, and a fifth utterance corresponding to the fourth phoneme.
19. The method of claim 18 , further comprising retrieving, from the word-corpus, the first utterance, the second utterance, the third utterance, the fourth utterance, and the fifth utterance.