Task assistant
A method of providing a task assistant to provide an interface to an application is described. The method comprises receiving input from a user through multimodal input including a plurality of speech input, typing input, and touch input, interpreting the input, and providing a formatted query to the application, receiving data from the application in response to the query, and providing a response to the user through multimodal output including a plurality of: speech output, text output, non-speech audio output, haptic output, and visual non-text output.
1. A method of providing a task assistant to provide an interface to an application, the method comprising:
activating, by a first computing device and for a user, a task assistant, wherein a previous activation of the task assistant has already occurred on a second computing device;
receiving, by the first computing device and from the second computing device, context data associated with the task assistant, wherein the context data is based on, for the user, the previous activation of the task assistant by the second computing device;
receiving, by the task assistant and from the user, speech input;
translating, based on the context data, the speech input to a query;
initiating a timer that has a time period based upon the previous activation of the task assistant, and closing the task assistant when the task assistant is not accessed prior to expiration of the time period; and
deleting the context data used to translate the speech input based upon, at least in part, determining that access of the task assistant did not occur prior to expiration of the time period.
2. The method of claim 1 , wherein a response to the query is provided through an overlay, displayed on top of a display of the application.
3. The method of claim 2 , wherein the task assistant is displayed as a semi-transparent overlay on the application.
4. The method of claim 1 , further comprising displaying the speech input to the user, such that information about the query is provided with a response to the query.
5. The method of claim 4 , further comprising: enabling the user to interact with the input displayed, to modify the query.
6. The method of claim 1 , further comprising: providing a soft invocation for the task assistant.
7. The method of claim 6 , wherein the soft invocation comprises one of: a button in a menu bar of the application, a button in a title bar of the application, a button in a header space of the application, a transparent ghost button floating over standard display of the application, a touch-input pattern, a movement input pattern, and an audio wake-up command.
8. The method of claim 1 , further comprising determining whether the input is unambiguous, and when the input is complete, generating the formatted query for the application.
9. The method of claim 8 , further comprising: when the input is not unambiguous, requesting clarification from the user, to enable construction of a complete query.
10. An apparatus comprising:
one or more processors; and
memory storing executable instructions, that, when executed by the one or more processors, cause the apparatus to perform operations comprising:
activating, by a first computing device and for a user, a task assistant, wherein a previous activation of the task assistant has already occurred on a second computing device;
receiving, by the first computing device and from the second computing device, context data associated with the task assistant, wherein the context data is based on, for the user, the previous activation of the task assistant by the second computing device;
receiving, by the task assistant and from the user, speech input;
translating, based on the context data, the speech input to a query;
initiating a timer that has a time period based upon the previous activation of the task assistant, and closing the task assistant when the task assistant is not accessed prior to expiration of the time period; and
deleting the context data used to translate the speech input based upon, at least in part, determining that access of the task assistant did not occur prior to expiration of the time period.
11. The task assistant of claim 10 , wherein a response to the query is provided through an overlay, displayed on top of a display of the application.
12. The task assistant of claim 10 , wherein the task assistant is displayed as a semi-transparent overlay on the application.
13. The task assistant of claim 10 , further comprising displaying the speech input to the user, such that information about the query is provided with a response to the query.
14. The task assistant of claim 13 , further comprising: an input user interface to enable the user to interact with the input displayed, to modify the query.
15. The task assistant of claim 10 , further comprising: a soft invoke logic to provide a soft invocation for the task assistant.
16. The task assistant of claim 15 , wherein the soft invocation comprises one of: a button in a menu bar of the application, a button in a title bar of the application, a button in a header space of the application, a transparent ghost button floating over standard display of the application, a touch-input pattern, a movement input pattern, and an audio wake-up command.
17. The task assistant of claim 10 , further comprising determining whether the input is unambiguous, and when the input is complete, generating the formatted query for the application.
18. The task assistant of claim 17 , further comprising: when the input is not unambiguous, requesting clarification from the user, to enable construction of a complete query.
19. One or more non-transitory computer-readable media, storing executable instructions that, when executed, cause an apparatus to perform operations comprising
activating, by a first computing device and for a user, a task assistant, wherein a previous activation of the task assistant has already occurred on a second computing device;
receiving, by the first computing device and from the second computing device, context data associated with the task assistant, wherein the context data is based on, for the user, the previous activation of the task assistant by the second computing device;
receiving, by the task assistant and from the user, speech input;
translating, based on the context data, the speech input to a query;
initiating a timer that has a time period based upon the previous activation of the task assistant, and closing the task assistant when the task assistant is not accessed prior to expiration of the time period; and
deleting the context data used to translate the speech input based upon, at least in part, determining that access of the task assistant did not occur prior to expiration of the time period.
20. The task assistant system of claim 19 , further comprising: a soft invoke logic to provide a soft invocation of a client application.