System and method for generating user models from transcribed dialogs
View Patent ↗Disclosed herein are systems, computer-implemented methods, and computer-readable storage media for generating personalized user models. The method includes receiving automatic speech recognition (ASR) output of speech interactions with a user, receiving an ASR transcription error model characterizing how ASR transcription errors are made, generating guesses of a true transcription and a user model via an expectation maximization (EM) algorithm based on the error model and the respective ASR output where the guesses will converge to a personalized user model which maximizes the likelihood of the ASR output. The ASR output can be unlabeled. The method can include casting speech interactions as a dynamic Bayesian network with four variables: (s), (u), (r), (m), and encoding relationships between (s), (u), (r), (m) as conditional probability tables. At each dialog turn (r) and (m) are known and (s) and (u) are hidden.
1. A method comprising:
receiving automatic speech recognition output of a plurality of speech interactions with a user;
receiving an automatic speech recognition transcription error model characterizing how automatic speech recognition transcription errors are made;
generating, via a processor, guesses of a true transcription and a user model via an expectation maximization algorithm, wherein the expectation maximization algorithm is based on the error model and the automatic speech recognition output; and
generating a personalized user model associated with a user voiceprint of the user based on the guesses and the user model.
2. The method of claim 1 , wherein generating the guesses of the true transcription and the user model further comprises iteratively repeating the following steps until a threshold is met:
guessing the true transcription from a current guess of the user model, to yield a current guess of the true transcription; and
guessing the user model based on the current guess of the true transcription.
3. The method of claim 1 , wherein the expectation maximization algorithm estimates conditional probabilities of hidden variables.
4. The method of claim 1 , wherein generating the guesses is further based on a set of manual transcriptions of speech interactions with the user.
5. The method of claim 4 , wherein the set of manual transcriptions is less numerous than the automatic speech recognition output.
6. The method of claim 1 , the method further comprising casting the speech interactions as a dynamical Bayesian network having variables and encoding relationships between the variables as conditional probability tables.
7. The method of claim 6 , wherein the variables comprise a user state, a user action, a speech recognition output, and a dialog system action.
8. The method of claim 1 , the method further comprising generating a personalized automatic speech recognition model based on the personalized user model.
9. The method of claim 1 , the method further comprising recognizing additional speech from the user based on the personalized speech model.
10. The method of claim 9 , the method further comprising iteratively improving the personalized speech model based on the additional speech.
11. The method of claim 1 , wherein the personalized user model applies to one of an individual user, a group of individual users and a population segment of.
12. The method of claim 11 , wherein the personalized user model applied to the segment of similar users is based on an individual user simulation.
13. The method of claim 1 , wherein the automatic speech recognition output is unlabeled.
14. A system comprising:
a processor; and
a computer-readable storage device having instructions stored which, when executed on the processor, perform operations comprising:
receiving automatic speech recognition output of a plurality of speech interactions with a user;
receiving an automatic speech recognition transcription error model characterizing how automatic speech recognition transcription errors are made;
generating, via a processor, guesses of a true transcription and a user model via an expectation maximization algorithm, wherein the expectation maximization algorithm is based on the error model and the respective automatic speech recognition output; and
generating a personalized user model associated with a user voiceprint of the user based on the guesses and the user model.
15. The system of claim 14 , wherein the automatic speech recognition output is unlabeled.
16. The system of claim 14 , wherein generating the guesses of the true transcription and the user model further comprises repeating the following steps until a threshold is met:
guessing the true transcription from a current guess of the user model, to yield a current guess of the true transcription; and
guessing the user model from the current guess of the true transcription.
17. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
receiving a user model personalized for a specific user generated by steps comprising:
receiving automatic speech recognition output of a plurality of speech interactions with the specific user;
receiving an automatic speech recognition transcription error model characterizing how automatic speech recognition transcription errors are made;
generating, via a processor, guesses of a true transcription and a user model via an expectation maximization algorithm, wherein the expectation maximization algorithm is based on the error model and the automatic speech recognition output; and
generating a personalized user model for the specific user based on the guesses and the user model; and
building a personalized dialog system for the specific user based associated with a user voiceprint of the user on the received personalized user model.
18. The computer-readable storage device of claim 17 , wherein the automatic speech recognition output is unlabeled.