Hierarchical speech recognition decoder
A speech interpretation module interprets the audio of user utterances as sequences of words. To do so, the speech interpretation module parameterizes a literal corpus of expressions by identifying portions of the expressions that correspond to known concepts, and generates a parameterized statistical model from the resulting parameterized corpus. When speech is received the speech interpretation module uses a hierarchical speech recognition decoder that uses both the parameterized statistical model and language sub-models that specify how to recognize a sequence of words. The separation of the language sub-models from the statistical model beneficially reduces the size of the literal corpus needed for training, reduces the size of the resulting model, provides more fine-grained interpretation of concepts, and improves computational efficiency by allowing run-time incorporation of the language sub-models.
1. A computer-implemented method, comprising:
receiving, over a computer network, an utterance of a user, the utterance having been accepted from the user at a client device as spoken input;
storing the utterance, the storing comprising identifying a plurality of sub-expressions by applying a parameterized statistical model that determines likely n-grams of literal word tokens and concept placeholders included in the utterance and storing each of the sub-expressions in the data structure as either: a set of literal word tokens representing the sub-expression, or a concept placeholder representing the sub-expression and providing an indication of a language sub-model;
determining likely textual representations of the sub-expressions stored as concept placeholders by applying the indicated language sub-models to the sub-expressions;
generating a user-specific textual interpretation of the utterance, the textual interpretation being a combination of the literal word tokens and the determined likely textual representations of the sub-expressions.
2. The computer-implemented method of claim 1 , wherein applying a language sub-model to a sub-expression to determine a likely textual representation of the sub-expression comprises:
identifying one or more additional language sub-models to apply to portions of the sub-expression;
accessing the one or more additional language sub-models; and
applying the accessed additional language sub-models to the associated portions of the sub-expression to determine likely textual representations of the portions of the sub-expression.
3. The computer-implemented method of claim 1 , wherein applying the indicated language sub-model comprises compiling the indicated language sub-model at runtime.
4. The computer-implemented method of claim 1 , wherein a language sub-model is explicitly specified based on rules of a logical grammar.
5. The computer-implemented method of claim 1 , wherein a language sub-model is a statistical language model trained from an input corpus.
6. The computer-implemented method of claim 5 , wherein the input corpus is a parameterized speech recognition corpus.
7. The computer-implemented method of claim 5 , wherein a likely textual representation of the sub-expression is selected by the statistical language model from a set of possible textual representations based on a likelihood of each textual representation being a correct textual interpretation of the sub-expression, wherein each of the possible textual representations in the set includes a weighted value, the weighted values indicating relative probabilities that the possible textual representations will appear in a user utterance.
8. A non-transitory computer-readable storage medium storing computer program instructions executable by a processor to perform operations comprising:
receiving, over a computer network, an utterance of a user, the utterance having been accepted from the user at a client device as spoken input;
storing the utterance, the storing comprising identifying a plurality of sub-expressions by applying a parameterized statistical model that determines likely n-grams of literal word tokens and concept placeholders included in the utterance and storing each of the sub-expressions in the data structure as either: a set of literal word tokens representing the sub-expression, or a concept placeholder representing the sub-expression and providing an indication of a language sub-model;
determining likely textual representations of the sub-expressions stored as concept placeholders by applying the indicated language sub-models to the sub-expressions;
generating a user-specific textual interpretation of the utterance, the textual interpretation being a combination of the literal word tokens and the determined likely textual representations of the sub-expressions.
9. The non-transitory computer-readable storage medium of claim 8 , wherein applying a language sub-model to a sub-expression to determine a likely textual representation of the sub-expression comprises:
identifying one or more additional language sub-models to apply to portions of the sub-expression;
accessing the one or more additional language sub-models; and
applying the accessed additional language sub-models to the associated portions of the sub-expression to determine likely textual representations of the portions of the sub-expression.
10. The non-transitory computer-readable storage medium of claim 8 , wherein applying the indicated language sub-model comprises compiling the indicated language sub-model at runtime.
11. The non-transitory computer-readable storage medium of claim 8 , wherein a language sub-model is explicitly specified based on rules of a logical grammar.
12. The non-transitory computer-readable storage medium of claim 8 , wherein a language sub-model is a statistical language model trained from an input corpus.
13. The non-transitory computer-readable storage medium of claim 12 , wherein the input corpus is a parameterized speech recognition corpus.
14. The non-transitory computer-readable storage medium of claim 12 , wherein a likely textual representation of the sub-expression is selected by the statistical language model from a set of possible textual representations based on a likelihood of each textual representation being a correct textual interpretation of the sub-expression, wherein each of the possible textual representations in the set includes a weighted value, the weighted values indicating relative probabilities that the possible textual representations will appear in a user utterance.
15. A system comprising:
a processor for executing computer program instructions; and
a non-transitory computer-readable storage medium storing computer program instructions executable by the processor to perform operations comprising:
receiving, over a computer network, an utterance of a user, the utterance having been accepted from the user at a client device as spoken input;
storing the utterance, the storing comprising identifying a plurality of sub-expressions by applying a parameterized statistical model that determines likely n-grams of literal word tokens and concept placeholders included in the utterance and storing each of the sub-expressions in the data structure as either: a set of literal word tokens representing the sub-expression, or a concept placeholder representing the sub-expression and providing an indication of a language sub-model;
determining likely textual representations of the sub-expressions stored as concept placeholders by applying the indicated language sub-models to the sub-expressions;
generating a user-specific textual interpretation of the utterance, the textual interpretation being a combination of the literal word tokens and the determined likely textual representations of the sub-expressions.
16. The system of claim 15 , wherein applying a language sub-model to a sub-expression to determine a likely textual representation of the sub-expression comprises:
identifying one or more additional language sub-models to apply to portions of the sub-expression;
accessing the one or more additional language sub-models; and
applying the accessed additional language sub-models to the associated portions of the sub-expression to determine likely textual representations of the portions of the sub-expression.
17. The system of claim 15 , wherein applying the indicated language sub-model comprises compiling the indicated language sub-model at runtime.
18. The system of claim 15 , wherein a language sub-model is explicitly specified based on rules of a logical grammar.
19. The system of claim 15 , wherein a language sub-model is a statistical language model trained from an input corpus.
20. The computer-implemented method of claim 19 , wherein a likely textual representation of the sub-expression is selected by the statistical language model from a set of possible textual representations based on a likelihood of each textual representation being a correct textual interpretation of the sub-expression, wherein each of the possible textual representations in the set includes a weighted value, the weighted values indicating relative probabilities that the possible textual representations will appear in a user utterance.