HMM DECODING COMPENSATION FOR SPEECH RECOGNITION AND MULTI-STRUCTURED DECODING FOR LOW RESOURCE COMMAND RECOGNITION
Described are techniques to recognize spoken wake word (WW) or command for human-machine interface using a speech recognition system that does not require any WW/command-matching speech data for training. The system uses the text or grapheme representation of the WW or commands for training before deployment. The technique includes receiving a target phrase for recognition by a speech recognition model. The technique includes analyzing a sequence of acoustic units representative of the target phrase when the target phrase is spoken to generate offline analysis data. The technique further includes constructing the speech recognition model based on the offline analysis data to decode speech signals of the target phrase according to the acoustic units. The technique further includes processing speech based on the speech recognition model to detect a presence of the target phrase.
1 . A method of speech recognition by a device, the method comprising:
receiving a target phrase for recognition by a speech recognition model;
analyzing a sequence of acoustic units representative of the target phrase when the target phrase is spoken to generate offline analysis data;
constructing the speech recognition model based on the offline analysis data to decode speech signals of the target phrase according to the acoustic units; and
processing speech based on the speech recognition model to detect a presence of the target phrase.
2 . The method of claim 1 , wherein processing speech based on the speech recognition model comprises:
determining from the speech recognition model a model likelihood score representing a likelihood of the presence of the target phrase based on an observed sequence of acoustic units decoded from the speech.
3 . The method of claim 2 , wherein processing speech based on the speech recognition model further comprises:
modifying the model likelihood score based on the observed sequence of acoustic units and the offline analysis data to determine the presence of the target phrase.
4 . The method of claim 1 , wherein the speech recognition model comprises a sequence of decoding states, wherein each decoding state of the sequence of decoding states models each acoustic unit of the sequence of acoustic units, and wherein analyzing the sequence of acoustic units comprises:
determining an order of transitioning through the sequence of decoding states based on time.
5 . The method of claim 1 , wherein the target phrase comprises a plurality of words, and wherein the offline analysis data comprises an expected length in time of the target phrase and an expected length in time of each of the plurality of words when the target phrase is spoken.
6 . The method of claim 1 , wherein the speech recognition model comprises a sequence of states, wherein each state of the sequence of states models each acoustic unit of the sequence of acoustic units, and wherein processing speech based on the speech recognition model comprises determining a likelihood of the presence of the target phrase based on an order of transitions between states of the sequence of states when the target phrase is spoken.
7 . The method of claim 1 , wherein the offline analysis data comprises an expected length in time of each of the acoustic units in the sequence of acoustic units.
8 . The method of claim 7 , wherein the speech recognition model comprises a sequence of decoding states, wherein each decoding state of the sequence of decoding states models each acoustic unit of the sequence of acoustic units, and wherein the expected length in time of each of the acoustic units comprises an expected length in time the speech recognition model stays in each of the decoding states when decoding speech signals of the target phrase.
9 . The method of claim 1 , wherein the speech recognition model comprises a sequence of decoding states, wherein each decoding state of the sequence of decoding states models each acoustic unit of the sequence of acoustic units, and wherein the offline analysis data comprises:
one or more acoustically similar acoustic units to an acoustic unit modeled by a decoding state, wherein the acoustically similar acoustic units are associated with probability estimates of a presence of the acoustically similar acoustic units when the decoding state identifies the acoustic unit modeled by the decoding state as a most likely acoustic unit.
10 . The method of claim 1 , wherein constructing the speech recognition model based on the offline analysis data comprises:
constructing a sequence decoding model based on the sequence of acoustic units, wherein the sequence decoding model includes a sequence of states, and wherein each state of the sequence of states models each acoustic unit of the sequence of acoustic units; and
constructing a decoding compensation model based on the offline analysis data to modify a decoding output of the sequence decoding model.
11 . The method of claim 10 , wherein the sequence decoding model decodes a most likely path through the sequence of states when processing speech, and wherein the decoding compensation model compares transitions through the sequence of states of the most likely path with expected transitions through the sequence of states when the speech recognition model processes acoustic units of the target phrase.
12 . The method of claim 11 , wherein the expected transitions through the sequence of states comprises at least one of:
a ratio between an expected length in time of a word in the target phrase and an expected total length in time of the target phrase when the target phrase is spoken;
an expected transition of 1 state through the sequence of states when the target phrase is spoken;
an expected length in time in each state of the sequence of states when the target phrase is spoken; or
acoustically similar acoustic units to an acoustic unit that is modeled by each state of the sequence of states, wherein each of the acoustically similar acoustic units is associated with a probability estimate of a detection when the acoustic unit is modeled by a corresponding state.
13 . The method of claim 12 , wherein processing speech based on the speech recognition model comprises at least one of:
comparing a ratio of an observed length in time of a word in the speech and an observed total length in time of the speech when transitioning through the sequence of states of the most likely path with the ratio between an expected length in time of a word in the target phrase and an expected total length in time of the target phrase to generate a word-ratio penalty;
comparing observed state jumps when transitioning through the sequence of states of the most likely path with the expected transition of 1 state for the target phrase to generate a state jump penalty;
comparing an observed length in time in each state when transitioning through the sequence of states of the most likely path with the expected length in time in each state of the target phrase to generate a state walk penalty; or
comparing a probability estimate of a most likely acoustic unit modeled by each state when transitioning through the sequence of states of the most likely path with probability estimates for the acoustically similar acoustic units and the acoustic unit modeled by each state for the expected transitions of the target phrase to generate a top-1 penalty.
14 . The method of claim 13 , wherein the state walk penalty is weighted by a probability of transitioning within each state for the expected transitions of the target phrase.
15 . The method of claim 13 , wherein the top-1 penalty for a state is weighted by a probability estimate of the acoustic unit modeled by the state to reward the most likely acoustic unit that matches the acoustic unit modeled by the state, and to penalize the most likely acoustic unit that fails to match the acoustic unit modeled by the state, when a probability estimate of the acoustic unit modeled by the state is high.
16 . The method of claim 13 , wherein processing speech based on the speech recognition model comprises:
combining the word-ratio penalty, the state jump penalty, the state walk penalty, and the top-1 penalty to generate a total compensation; and
modifying a score associated with the most likely path by the total compensation to generate a modified score indicating a probability of the presence of the target phrase.
17 . The method of claim 1 , wherein the target phrase comprises at least one of:
a wake-word spoken to address the device;
a simple command spoken following the wake-word, wherein the simple command includes one or more words;
a compound command spoken following the wake-word, wherein the compound command includes a common sub-command and a second sub-command unique to each compound command;
a number and an associated unit spoken following the wake-word; or
a complex command spoken following the wake-word, wherein the complex command includes a combination of any one of the simple command, the compound command, and the number and the associated unit.
18 . The method of claim 17 , wherein constructing the speech recognition model based on the offline analysis data comprises:
constructing a sequence decoding model based on a sequence of acoustic units of the wake-word followed by a sequence of acoustic units of a command, wherein the sequence decoding model models a first sequence of states corresponding to the sequence of acoustic units of the wake-word and a second sequence of states corresponding to the sequence of acoustic units of the command, and wherein a state of the first sequence of states corresponding to a last acoustic unit of the wake-word also models a gap between the wake-word and the command.
19 . The method of claim 17 , wherein constructing the speech recognition model based on the offline analysis data comprises:
constructing a sequence decoding model based on a concatenation of sequences of acoustic units of a plurality of words of a command, wherein the sequence decoding model includes a first sequence of states modeling a sequence of acoustic units of a first word, an silence state modeling a gap between the first word and a second word of the command, and a second sequence of states modeling a sequence of acoustic units of the second word, and wherein the silence state also models a last acoustic unit of the first word and a first acoustic unit of the second word.
20 . An apparatus comprising:
an input terminal configured to receive an audio signal from one or more microphones; and
a processing system configured to:
receive a target phrase for recognition by a speech recognition model;
analyze a sequence of acoustic units representative of the target phrase when the target phrase is spoken to generate offline analysis data;
construct the speech recognition model based on the offline analysis data to decode speech signals of the target phrase according to the acoustic units; and
process the audio signal based on the speech recognition model to detect a presence of the target phrase.