System and method of handling problematic input during context-sensitive help for multi-modal dialog systems
View Patent ↗A method of presenting a multi-modal help dialog move to a user in a multi-modal dialog system is disclosed. The method comprises presenting an audio portion of the multi-modal help dialog move that explains available ways of user inquiry and presenting a corresponding graphical action performed on a user interface associated with the audio portion. The multi-modal help dialog move is context-sensitive and uses current display information and dialog contextual information to present a multi-modal help move that is currently related to the user. A user request or a problematic dialog detection module may trigger the multi-modal help move.
1. In a multi-modal dialog system, a method of presenting a multi-modal help dialog move to a user in a problematic dialog situation, the method comprising:
receiving a plurality of utterances that do not meet a successful recognition threshold;
in response to the received plurality of utterances, presenting an audio portion of a multi-modal help dialog move that explains available non-audio ways of user inquiry; and
presenting a corresponding graphical action performed on a user interface associated with the audio portion.
2. The method of claim 1 , wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.
3. The method of claim 1 , wherein the presented graphical action further comprises at least one graphical button corresponding to choices available to the user.
4. The method of claim 3 , wherein the at least one graphical button relates to different kinds of help available to the user.
5. The method of claim 1 , wherein the number of the plurality of user utterances that does not meet the successful recognition threshold is adjustable based on user experience.
6. The method of claim 1 , wherein the number of the plurality of user utterances that does not meet the successful recognition threshold is adjustable based on automatic speech recognition performance.
7. The method of claim 1 , further comprising:
receiving at least one graphical user input that does not meet a graphical user input recognition threshold.
8. The method of claim 7 , wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.
9. The method of claim 7 , wherein the presented graphical action further comprises at least one graphical button corresponding to choices available to the user.
10. The method of claim 9 , wherein the at least one graphical button relates to different kinds of help available to the user.
11. The method of claim 1 , wherein the number of plurality of utterances is set according to user experience and automatic speech recognition parameters.
12. A multi-modal dialog system for presenting a multi-modal help dialog move to a user, in a problematic dialog situation, the system comprising:
a module receiving a plurality of utterances that do not meet a successful recognition threshold;
in response to the received plurality of utterances, a module presenting an audio portion of a multi-modal help dialog move that explains available non-audio ways of user inquiry; and
a module presenting a corresponding graphical action performed on a user interface associated with the audio portion.
13. The system of claim 12 , wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.
14. The system of claim 12 , wherein the presented graphical action further comprises at least one graphical button corresponding to choices available to the user.
15. The system of claim 14 , wherein the at least one graphical button relates to different kinds of help available to the user.
16. The system of claim 12 , wherein the number of the plurality of user utterances that does not meet the successful recognition threshold is adjustable based on user experience.
17. The system of claim 11 , wherein the number of the plurality of user utterances that does not meet the successful recognition threshold is adjustable based on automatic speech recognition performance.
18. A computer readable medium storing a computer program for controlling a computing device to present a multi-modal help dialog move to a user in a problematic dialog situation, the computer program having instructions comprising:
receiving a plurality of utterances that do not meet a successful recognition threshold;
in response to the received plurality of utterances, presenting an audio portion of a multi-modal help dialog move that explains available non-audio ways of user inquiry; and
presenting a corresponding graphical action performed on a user interface associated with the audio portion.
19. The computer readable medium of claim 18 , wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.
20. The computer readable medium of claim 18 , wherein the presented graphical action further comprises at least one graphical button corresponding to choices available to the user.