System and method of providing an automated data-collection in spoken dialog systems
The invention relates to a system and method for gathering data for use in a spoken dialog system. An aspect of the invention is generally referred to as an automated hidden human that performs data collection automatically at the beginning of a conversation with a user in a spoken dialog system. The method comprises presenting an initial prompt to a user, recognizing a received user utterance using an automatic speech recognition engine and classifying the recognized user utterance using a spoken language understanding module. If the recognized user utterance is not understood or classifiable to a predetermined acceptance threshold, then the method re-prompts the user. If the recognized user utterance is not classifiable to a predetermined rejection threshold, then the method transfers the user to a human as this may imply a task-specific utterance. The received and classified user utterance is then used for training the spoken dialog system.
1. A method comprising:
training a spoken dialog system using task-independent call-types of a previous application;
recognizing a user utterance using the spoken dialog system, to yield a recognized user utterance;
determining a rejection threshold based on an entity referenced in the recognized user utterance;
comparing a classification of the recognized user utterance to the rejection threshold, to yield a comparison; and
when the comparison indicates the recognized user utterance meets the rejection threshold, transcribing the recognized user utterance and using the recognized user utterance for further training of the spoken dialog system.
2. The method of claim 1 , wherein the user utterance comprises a task-specific call-type.
3. The method of claim 2 , wherein the spoken dialog system performs call-type classification resulting in the classification.
4. The method of claim 1 , further comprising issuing a prompt to the user to receive the user utterance a second time.
5. The method of claim 4 , further comprising counting a number of re-prompts.
6. The method of claim 5 , wherein when the number of re-prompts meets a re-prompt number, setting the classification to meet the rejection threshold.
7. The method of claim 1 , performed by an automated hidden human.
8. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
training a spoken dialog system using task-independent call-types of a previous application;
recognizing a user utterance using the spoken dialog system, to yield a recognized user utterance;
determining a rejection threshold based on an entity referenced in the recognized user utterance;
comparing a classification of the recognized user utterance to the rejection threshold, to yield a comparison; and
when the comparison indicates the recognized user utterance meets the rejection threshold, transcribing the recognized user utterance and using the recognized user utterance for further training of the spoken dialog system.
9. The system of claim 8 , wherein the user utterance comprises a task-specific call-type.
10. The system of claim 9 , wherein the spoken dialog system performs call-type classification resulting in the classification.
11. The system of claim 8 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations comprising issuing a prompt to the user to receive the user utterance a second time.
12. The system of claim 11 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations comprising counting a number of re-prompts.
13. The system of claim 12 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations, wherein when the number of re-prompts meets a re-prompt number, setting the classification to meet the rejection threshold.
14. The system of claim 8 , performed by an automated hidden human.
15. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
training a spoken dialog system using task-independent call-types of a previous application;
recognizing a user utterance using the spoken dialog system, to yield a recognized user utterance;
determining a rejection threshold based on an entity referenced in the recognized user utterance;
comparing a classification of the recognized user utterance to the rejection threshold, to yield a comparison; and
when the comparison indicates the recognized user utterance meets the rejection threshold, transcribing the recognized user utterance and using the recognized user utterance for further training of the spoken dialog system.
16. The computer-readable storage device of claim 15 , wherein the user utterance comprises a task-specific call-type.
17. The computer-readable storage device of claim 16 , wherein the spoken dialog system performs call-type classification resulting in the classification.
18. The computer-readable storage device of claim 17 , having additional instructions stored which, when executed by the computing device, result in operations comprising issuing a prompt to the user to receive the user utterance a second time.
19. The computer-readable storage device of claim 18 , having additional instructions stored which, when executed by the computing device, result in operations comprising counting a number of re-prompts.
20. The computer-readable storage device of claim 15 , having additional instructions stored which, when executed by the computing device, result in operations, wherein when the number of re-prompts meets a re-prompt number, setting the classification to meet the rejection threshold.