System and method for using meta-data dependent language modeling for automatic speech recognition
View Patent ↗Disclosed are systems and methods for providing a spoken dialog system using meta-data to build language models to improve speech processing. Meta-data is generally defined as data outside received speech; for example, meta-data may be a customer profile having a name, address and purchase history of a caller to a spoken dialog system. The method comprises building tree clusters from meta-data and estimating a language model using the built tree clusters. The language model may be used by various modules in the spoken dialog system, such as the automatic speech recognition module and/or the dialog management module. Building the tree clusters from the meta-data may involve generating projections from the meta-data and further may comprise computing counts as a result of unigram tree clustering and then building both unigram trees and higher-order trees from the meta-data as well as computing node distances within the built trees that are used for estimating the language model.
1. A method of generating a language model using non-speech meta-data, the method comprising:
applying a tree growing algorithm to highly fragmented non-speech meta-data related to a caller to a spoken dialog system but does not describe physical characteristics of the caller;
identifying a first quantity of leaf nodes in which a history appears as a result of applying the tree growing algorithm;
identifying a second quantity of projections based on the highly fragmented non-speech meta-data, wherein the first quantity equals the second quantity; and
estimating a conditional meta-data dependent language model using the identified projections and speech data.
2. The method of claim 1 , wherein identifying the second quantity of projections based on the highly fragmented non-speech meta-data further comprises using divisive likelihood-based techniques to identify the non-speech meta-data projections.
3. The method of claim 2 , wherein identifying the second quantity of projections based on the highly fragmented non-speech meta-data further comprises using tree-based clustering to identify the non-speech meta-data projections.
4. The method of claim 1 , wherein the conditional language model is estimated by merging non-speech meta-data conditional counts that are weighted based on a given non-speech meta-data value.
5. The method of claim 1 , wherein the estimated language model is used for automatic speech recognition.
6. The method of claim 1 , wherein identifying the second quantity of projections based on non-speech meta-data is performed at training time.
7. The method of claim 6 , wherein estimating a conditional language model using the identified projections is performed at test time.
8. The method of claim 1 , wherein identifying the second quantity of projections based on non-speech meta-data further comprises estimating a unigram model to generate a unigram clustering output.
9. The method of claim 8 , wherein the unigram clustering output is used for building a higher order tree.
10. The method of claim 9 , wherein building the higher order tree further comprises:
generating history nodes for the higher order tree; and
splitting leaves on the higher order tree via domain splits.
11. The method of claim 10 , wherein generating history nodes for the higher order tree further comprises:
partitioning count distributions allowing only history dependent splits; and
terminating the splitting based on a threshold value.
12. The method of claim 9 , further comprising:
computing a distance between a root node of a unigram tree and at least one history node of a higher order tree.
13. The method of claim 12 , wherein estimating a conditional language model using the identified projections further comprises weighing the identified projections using the computed distance.
14. The method of claim 8 , wherein the step of estimating a unigram model to generate a unigram clustering output further comprises:
generating a unigram tree by splitting a leaf with the largest likelihood gain; and
evaluating the likelihood gain of new leaf created by the splits.
15. The method of claim 14 , wherein splitting the leaf with the largest likelihood gain is iterative.
16. The method of claim 15 , wherein one the likelihood gain of a best leaf split falls below a threshold value, terminating the iterative splitting process, wherein the unigram tree defines N leaf sets.
17. The method of claim 1 , wherein estimating a conditional language model using the identified projections further comprises estimating the conditional language model by estimating history dependent distributions for histories in training data.
18. The method of claim 1 , where the highly fragmented non-speech meta-data further comprises a name, address and purchase history of the caller to the spoken dialog system.
19. A method of building a language model used for automatic speech recognition (ASR), the method comprising:
applying a tree growing algorithm to highly fragmented non-speech meta-data related to a caller to a spoken dialog system but does not describe physical characteristics of the caller;
identifying a first quantity of leaf nodes in which a history appears as a result of applying the tree growing algorithm;
building tree clusters from non-speech meta-data by generating a second quantity of projections based on the highly fragmented non-speech meta-data, wherein the first quantity equals the second quantity; and
estimating a meta-data dependent language model using the built tree clusters and speech data, the language model used for ASR.
20. The method of claim 19 , wherein building tree clusters from non-speech meta-data further comprises:
computing counts from the non-speech meta-data;
building a unigram tree from the computed counts;
building at least one higher-order tree using the built unigram tree; and
computing node distances, wherein the computed node distances are used to estimate the language model.
21. The method of claim 19 , wherein ASR is used in a spoken dialog system.
22. The method of claim 19 , wherein different language models are estimated based on different types of data within the non-speech meta-data.
23. The method of claim 19 , wherein the language model is estimated by weighing non-speech meta-data towards a type of data within the non-speech meta-data.
24. The method of claim 23 , wherein the non-speech meta-data includes gender information and is weighed towards one gender.
25. A tangible computer-readable medium storing instructions for controlling a computing device to operate an automatic speech recognition (ASR) module using a language model, the language model generated by the steps:
applying a tree growing algorithm to highly fragmented non-speech meta-data related to a caller to a spoken dialog system but does not describe physical characteristics of the caller;
identifying a first quantity of leaf nodes in which a history appears as a result of applying the tree growing algorithm;
building tree clusters from non-speech meta-data by generating a second quantity of projections based on the highly fragmented non-speech meta-data, wherein the first quantity equals the second quantity; and
estimating the meta-data dependent language model using the built tree clusters and speech data, the language model used by the ASR module.
26. The tangible computer-readable medium of claim 25 , wherein building tree clusters from non-speech meta-data further comprises:
computing counts from the non-speech meta-data;
building a unigram tree from the computed counts;
building at least one higher-order tree using the built unigram tree; and
computing node distances, wherein the computed node distances are used to estimate the language model.
27. A spoken dialog system having a plurality of modules for processing speech, at least one module of the plurality of modules uses a language model estimated by the steps:
applying a tree growing algorithm to highly fragmented non-speech meta-data related to a caller to a spoken dialog system but does not describe physical characteristics of the caller;
identifying a first quantity of leaf nodes in which a history appears as a result of applying the tree growing algorithm;
building tree clusters from non-speech meta-data by generating a second quantity of projections based on the highly fragmented non-speech meta-data, wherein the first quantity equals the second quantity; and
estimating a meta-data dependent language model using the built tree clusters and speech data.
28. The spoken dialog system of claim 27 , wherein building tree clusters from non-speech meta-data further comprises:
computing counts from the non-speech meta-data;
building a unigram tree from the computed counts;
building at least one higher-order tree using the built unigram tree; and
computing node distances, wherein the computed node distances are used to estimate the language model.