Multi-state barge-in models for spoken dialog systems
View Patent ↗A method is disclosed for applying a multi-state barge-in acoustic model in a spoken dialogue system. The method includes receiving an audio speech input from the user during the presentation of a prompt, accumulating the audio speech input from the user, applying a non-speech component having at least two one-state Hidden Markov Models (HMMs) to the audio speech input from the user, applying a speech component having at least five three-state HMMs to the audio speech input from the user, in which each of the five three-state HMMs represents a different phonetic category, determining whether the audio speech input is a barge-in-speech input from the user, and if the audio speech input is determined to be the barge-in-speech input from the user, terminating the presentation of the prompt.
1. A method comprising:
receiving an input having a speech segment and a non-speech segment;
establishing a first restriction of recognizing only speech states during the speech segment;
establishing a second restriction of recognizing only non-speech states during the non-speech segment;
generating, via a processor, a hypothesis lattice, wherein the hypothesis lattice allows any sequence of speech states and non-speech states; and
generating a reference lattice, wherein the reference lattice is based on the hypothesis lattice and conforms to the first restriction and the second restriction.
2. The method of claim 1 , wherein the speech states and the non-speech states are Hidden Markov Models.
3. The method of claim 1 , further comprising employing the reference lattice as a barge-in model for speech processing.
4. The method of claim 3 , wherein the barge-in model is trained using a maximum likelihood training on data partitioned into the speech segment and the non-speech segment.
5. The method of claim 4 , wherein the maximum likelihood training determines maximum likelihood based on shared data between the speech segment and the non-speech segment.
6. The method of claim 1 , wherein the reference lattice is redefined at each iteration of training
7. The method of claim 1 , wherein of the first restriction and the second restriction allows for re-alignment due to mislabeling of the input.
8. A system comprising:
a processor; and
a storage device having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
receiving an input having a speech segment and a non-speech segment;
establishing a first restriction of recognizing only speech states during the speech segment;
establishing a second restriction of recognizing only non-speech states during the non-speech segment;
generating a hypothesis lattice, wherein the hypothesis lattice allows any sequence of speech states and non-speech states; and
generating a reference lattice, wherein the reference lattice is based on the hypothesis lattice and conforms to the first restriction and the second restriction.
9. The system of claim 8 , wherein the speech states and the non-speech states are Hidden Markov Models.
10. The system of claim 8 , wherein the storage device has additional instructions stored which result in the operations further comprising employing the reference lattice as a barge-in model for speech processing.
11. The system of claim 10 , wherein the barge-in model is trained using a maximum likelihood training on data partitioned into the speech segment and the non-speech segment.
12. The system of claim 11 , wherein the maximum likelihood training determines maximum likelihood based on shared data between the speech segment and the non-speech segment.
13. The system of claim 8 , wherein the reference lattice is redefined at each iteration of training
14. The system of claim 8 , wherein of the first restriction and the second restriction allows for re-alignment due to mislabeling of the input.
15. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
receiving an input having a speech segment and a non-speech segment;
establishing a first restriction of recognizing only speech states during the speech segment;
establishing a second restriction of recognizing only non-speech states during the non-speech segment;
generating a hypothesis lattice, wherein the hypothesis lattice allows any sequence of speech states and non-speech states; and
generating a reference lattice, wherein the reference lattice is based on the hypothesis lattice and conforms to the first restriction and the second restriction.
16. The computer-readable storage device of claim 15 , wherein the speech states and the non-speech states are Hidden Markov Models.
17. The non transitory computer-readable storage device of claim 15 , the computer readable storage device having additional instructions stored which result in the operations further comprising employing the reference lattice as a barge-in model for speech processing.
18. The computer-readable storage device of claim 17 , wherein the barge-in model is trained using a maximum likelihood training on data partitioned into the speech segment and the non-speech segment.
19. The computer-readable storage device of claim 18 , wherein the maximum likelihood training determines maximum likelihood based on shared data between the speech segment and the non-speech segment.
20. The computer-readable storage device of claim 15 , wherein the reference lattice is redefined at each iteration of training.