LANGUAGE INDEPENDENT DICTIONARY-TRAINED GRAPHEME-TO-PHONEME CONVERTER AND TEXT-TO-SPEECH ENGINE FOR IMPROVED SPEECH RECOGNITION
Described are techniques to recognize spoken wake word (WW) or command for human-machine interface using a speech recognition system that does not require any WW/command-matching speech data for training. The technique trains a tokenizer and includes breaking a word from a database into a plurality of combinations of unique sub-words by splitting the word at a plurality of different points for each of the combinations. The word includes one or more written units and one or more corresponding acoustic units. The technique maps the acoustic units constituting the word to the written units constituting the word to generate an acoustic-unit-to-written-unit mapping. The technique includes assigning a subset of the acoustic units to each of the unique sub-words based on the acoustic-unit-to-written-unit mapping to generate an acoustic-units-to-sub-words assignment for the word. The technique accumulates the acoustic-units-to-sub-words assignment for a plurality of words from the database to create a sub-word likelihoods dictionary.
1 . A method of training a tokenizer to generate a string of acoustic units from input text, the method comprising:
breaking a word from a database into a plurality of combinations of unique sub-words by splitting the word at a plurality of different points for each of the combinations, wherein the word includes one or more written units and one or more corresponding acoustic units;
mapping the acoustic units constituting the word to the written units constituting the word to generate an acoustic-unit-to-written-unit mapping;
assigning a subset of the acoustic units to each of the unique sub-words based on the acoustic-unit-to-written-unit mapping to generate an acoustic-units-to-sub-words assignment for the word; and
accumulating the acoustic-units-to-sub-words assignment for a plurality of words from the database to create a sub-word likelihoods dictionary.
2 . The method of claim 1 , wherein the acoustic-unit-to-written-unit mapping comprises at least one of:
a mapping of an acoustic unit to a written unit;
a mapping of a plurality of acoustic units to a written unit; or
a mapping of a plurality of written units to an acoustic unit.
3 . The method of claim 1 , wherein the acoustic-units-to-sub-words assignment for the word is based on locations of the points used to split the word into the combinations of unique sub-words.
4 . The method of claim 1 , wherein each of the sub-words comprises a plurality of acoustic units, and wherein the sub-word likelihoods dictionary compiles a probability of a unique combination of acoustic units for each of the sub-words.
5 . The method of claim 4 , wherein the probability of a unique combination of acoustic units for each of the sub-words is a function of locations of the sub-words within a word.
6 . The method of claim 4 , further comprising:
invoking the tokenizer to synthesize an input text of a target phrase into a string of acoustic units representing the target phrase based on the sub-word likelihoods dictionary.
7 . The method of claim 6 , wherein invoking the tokenizer comprises:
breaking the input text of the target phrase into a plurality of unique sub-word splits;
searching through the sub-word likelihoods dictionary for each sub-word of the unique sub-word splits to determine a probability of the unique sub-words splits, wherein the unique sub-word splits has a phonetic solution corresponding to a combination of an acoustic unit for each sub-word of the unique sub-word splits; and
analyzing the probability of the unique sub-words splits for the plurality of unique sub-word splits to generate one or more most likely phonetic solutions for the target phrase.
8 . The method of claim 7 , wherein the probability of the unique sub-word splits comprises a product of the probability for each sub-word of the unique sub-word splits obtained from the sub-word likelihoods dictionary.
9 . The method of claim 7 , wherein the string of acoustic units representing the target phrase comprises the combination of the acoustic unit for each sub-word of a unique sub-word splits corresponding to a most likely phonetic solution of the one or more most likely phonetic solutions for the target phrase.
10 . The method of claim 7 , further comprising:
processing speech based on the one or more most likely phonetic solutions for the target phrase to detect a presence of the target phrase in the speech.
11 . The method of claim 1 , further comprising:
tuning a text-to-speech (TTS) model to generate synthetic speech from input text based on an annotated database of real speech data and corresponding words, wherein the annotated database lacks real speech data corresponding to a target phrase;
generating synthetic speech of the target phrase based on the TTS model;
analyzing the synthetic speech of the target phrase to compile statistics on acoustic units; and
processing speech aided by the statistics to detect a presence of the target phrase in the speech.
12 . The method of claim 11 , wherein the TTS model comprises parameters to control generating synthetic speech from input text, and wherein tuning the TTS model comprises:
generating synthetic speech data of a plurality of words from the annotated database based on the parameters;
processing the synthetic speech data to generate vectors characterizing the synthetic speech data;
processing the real speech data corresponding to the plurality of words to generate vectors characterizing the real speech data;
comparing the vectors characterizing the synthetic speech data and the vectors characterizing the real speech data to generate comparison results; and
adjusting iteratively the parameters of the TTS model based on the comparison results to generate a compensation model.
13 . The method of claim 12 , wherein the parameters comprise at least one of:
settings of the TTS model;
characteristics of speakers of the synthetic speech data; or
parameters for augmenting the synthetic speech data.
14 . The method of claim 13 , wherein the parameters for augmenting the synthetic speech data comprises at least one of:
parameters for modifying a time scale of the synthetic speech data;
parameters for modifying spectral content of the synthetic speech data; or
gain parameters for the synthetic speech data.
15 . The method of claim 12 , wherein generating synthetic speech of the target phrase based on the TTS model comprises:
generating the synthetic speech of the target phrase based on the compensation model.
16 . The method of claim 11 , wherein analyzing the synthetic speech of the target phrase to compile statistics on acoustic units comprises:
determining time boundaries of each acoustic unit of the target phrase based on the input text of the target phrase;
analyzing spectral and temporal characteristics of each acoustic unit to generate observation vectors;
generating a most likely acoustic unit within the time boundaries of each acoustic unit based on the observation vectors; and
analyzing the most likely acoustic unit within the time boundaries of each of a plurality of acoustic units of synthetic speech for a plurality of target phrases to generate a probability of predicting one of a plurality of most likely acoustic units when processing a given acoustic unit of the speech.
17 . The method of claim 16 , wherein processing speech aided by the statistics comprises:
detecting alternate pronunciations of the target phrase by a speech recognition model based on the probability of predicting one of the plurality of most likely acoustic units when the target phrase is spoken.
18 . The method of claim 17 , wherein the speech recognition model comprises a sequence of decoding states, wherein each decoding state of the sequence of decoding states models each acoustic unit of the target phrase, and wherein one of the sequence of decoding states models a plurality of acoustic units to support alternate pronunciations of the target phrase based on the probability.
19 . The method of claim 11 , wherein analyzing the synthetic speech of the target phrase to compile statistics on acoustic units comprises:
determining time boundaries of each acoustic unit of the target phrase based on the input text of the target phrase;
analyzing spectral and temporal characteristics of each acoustic unit to generate observation vectors;
generating a most likely acoustic unit within the time boundaries of each acoustic unit based on the observation vectors; and
analyzing the most likely acoustic unit within the time boundaries of each acoustic unit of a plurality of acoustic units of synthetic speech for a plurality of target phrases to generate an average length in time of each acoustic unit.
20 . An apparatus to train a tokenizer to generate a string of acoustic units from input text, comprising:
a processing system configured to:
break a word from a database into a plurality of combinations of unique sub-words by splitting the word at a plurality of different points for each of the combinations, wherein the word includes one or more written units and one or more corresponding acoustic units;
map the acoustic units constituting the word to the written units constituting the word to generate an acoustic-unit-to-written-unit mapping;
assign a subset of the acoustic units to each of the unique sub-words based on the acoustic-unit-to-written-unit mapping to generate an acoustic-units-to-sub-words assignment for the word; and
accumulate the acoustic-units-to-sub-words assignment for a plurality of words from the database to create a sub-word likelihoods dictionary.