System and method of using meta-data in speech processing
View Patent ↗Systems and methods relate to generating a language model for use in, for example, a spoken dialog system or some other application. The method comprises building a class-based language model, generating at least one sequence network and replacing class labels in the class-based language model with the at least one sequence network. In this manner, placeholders or tokens associated with classes can be inserted into the models at training time and word/phone networks can be built based on meta-data information at test time. Finally, the placeholder token can be replaced with the word/phone networks at run time to improve recognition of difficult words such as proper names.
1. A method for generating a language model, the method comprising:
building a class-based language model;
generating, via a processor, at run time at least one sequence network based on side information that is different from any data used to build the class-based language model; and
replacing class labels in the class-based language model with the at least one sequence network, wherein the at least one sequence network comprises members of a class for each class label, the members of the class identified by the side information.
2. The method of claim 1 , wherein building the class-based language model further comprises replacing sequences of tokens from a training corpus for the class-based language model with the class labels.
3. The method of claim 2 , wherein the class labels relate to at least one of a mailbox name and a caller name.
4. The method of claim 2 , wherein the class labels are treated as words and building the class-based language model further comprises:
building a trigram model; and
encoding the trigram model as a weighted finite-state automaton.
5. The method of claim 4 , wherein transitions in the weighted finite-state automaton are replaced by sequences of words that are members of a class associated with the class labels.
6. The method of claim 1 , wherein generating at least one sequence network further comprises:
estimating for each class of the class-based language model a probability of different realizations of words in each class; and
replacing a first weighted sequence network with the different realizations of words in each class to generate a second weighted sequence network.
7. The method of claim 6 , wherein the different realization of the words associated with the first weighted sequence network and the second weighted sequence network are names.
8. The method of claim 1 , wherein the sequence network is a name sequence network.
9. The method of claim 1 , wherein replacing class labels in the class-based language model with the at least one sequence network further comprises combining the weights of the at least one sequence network.
10. The method of claim 9 , wherein combining the weights of the at least one sequence network is performed with a composition of finite-state transducers.
11. The method of claim 1 , wherein each class label is a name class label that has a special phone symbol in a lexicon.
12. The method of claim 11 , wherein replacing class labels in the class-based language model with the at least one sequence network further comprises:
combining the class-based language model with the lexicon;
optimizing the combined class-based language model and lexicon; and
replacing each transition in the class-based language model having a name class label as the output label with the optimized, combined class-based language model and lexicon for that name class.
13. The method of claim 12 , further comprising combining weights in the replacement of each transition.
14. The method of claim 1 , wherein building a class-based language model is performed at training time, generating at least one sequence network is performed at test time and replacing class labels in the class-based language model with the at least one sequence network is performed at run-time.
15. A speech recognition system, the speech recognition system comprising:
a first module configured to control a processor to build a class-based language model;
a second module configured to control the processor to generate at run time at least one sequence network based on side information that is different from any data used to build the class-based language model; and
a third module configured to control the processor to replace class labels in the class-based language model with the at least one sequence network, wherein the at least one sequence network comprises members of a class for each class label, the members of the class identified by the side information.
16. The speech recognition system of claim 15 , wherein the first module further replaces sequences of tokens from a training corpus for the class-based language model with the class labels.
17. The speech recognition system of claim 16 , wherein the class labels are treated as words and the first module further:
builds a trigram model; and
encodes the trigram model as a weighted finite-state automaton.
18. The speech recognition system of claim 17 , wherein transitions in the weighted finite-state automaton are replaced by sequences of words that are members of a class associated with the class labels.
19. The speech recognition system of claim 15 , wherein the sequence network is a name sequence network.
20. The speech recognition system of claim 15 , wherein the third module further combines the weights of the at least one sequence network.
21. The speech recognition system of claim 15 , wherein the first module builds the class-based language model at training time, the second module generates the at least one sequence network at test time and the first module replaces class labels in the class-based language model with the at least one sequence network at run-time.
22. A non-transitory computer-readable medium that stores instructions for controlling a computing device to generate a language model, the instructions comprising the steps:
building a class-based language model;
generating at run time at least one sequence network based on side information that is different from any data used to build the class-based language model; and
replacing class labels in the class-based language model with the at least one sequence network, wherein the at least one sequence network comprises members of a class for each class label, the members of the class identified by the side information.
23. The non-transitory computer-readable medium of claim 22 , wherein building the class-based language model further comprises replacing sequences of tokens from a training corpus for the class-based language model with the class labels.
24. The non-transitory computer-readable medium of claim 23 , wherein the class labels are treated as words and building the class-based language model further comprises:
building a trigram model; and
encoding the trigram model as a weighted finite-state automaton.
25. The non-transitory computer-readable medium of claim 24 , wherein transitions in the weighted finite-state automaton are replaced by sequences of words that are members of a class associated with the class labels.
26. The non-transitory computer-readable medium of claim 22 , wherein the class labels relate to at least one of a mailbox name and a caller name.
27. The non-transitory computer-readable medium of claim 22 , wherein generating at least one sequence network further comprises:
estimating for each class of the class-based language model a probability of different realizations of words in each class; and
replacing a first weighted sequence network with the different realizations of words in each class to generate a second weighted sequence network.
28. The non-transitory computer-readable medium of claim 27 , wherein the different realization of the words associated with the first weighted sequence network and the second weighted sequence network are names.
29. The non-transitory computer-readable medium of claim 22 , wherein the sequence network is a name sequence network.
30. The non-transitory computer-readable medium of claim 22 , wherein building a class-based language model is performed at training time, generating at least one sequence network is performed at test time and replacing class labels in the class-based language model with the at least one sequence network is performed at run-time.