Intelligent automated assistant
The intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.
1. A method for launching an application on a user device using a digital assistant, comprising:
at an electronic device comprising a processor and memory storing instructions for execution by the processor:
providing, at the user device, a graphical user interface including an at least partially speech-based conversational interface for interacting with the user, the graphical user interface displaying at least a portion of a conversational interaction between the user and the user device;
obtaining context information associated with an interaction between the user and the user device;
receiving a speech input from the user through the conversational interface;
processing the speech input to determine a user intent associated with the speech input; and
upon determination that the user intent associated with the speech input is for invoking a software application installed on the user device:
invoking the software application on the user device external to the graphical user interface including the conversational interface; and
providing a response based on the user intent and the context information.
2. The method of claim 1 , wherein obtaining the context information further comprises:
receiving an additional speech input through the conversational interface prior to receiving the speech input; and
processing the additional speech input to obtain the context information.
3. The method of claim 1 , wherein processing the speech input to determine the user intent associated with the speech input further comprises:
disambiguating the speech input using the context information.
4. The method of claim 1 , further comprising:
receiving an additional speech input from the user through the conversational interface;
processing the additional speech input to determine an additional user intent associated with the additional speech input; and
based on the additional user intent that has been determined:
obtaining another input related to the additional user intent from the user through the conversational interface; and
executing a task for fulfilling the additional user intent within the conversational interface.
5. The method of claim 1 , further comprising:
receiving an additional speech input from the user through the conversational interface;
processing the additional speech input to determine an additional user intent associated with the additional speech input;
upon determination that the additional user intent associated with the additional speech input is for invoking an additional software application installed on the user device:
invoking the additional software application on the user device outside of the conversational interface.
6. The method of claim 1 , wherein obtaining the context information further comprises:
receiving non-speech input from the user prior to the speech input; and
processing the non-speech input to obtain the context information.
7. A system, comprising:
one or more processors;
memory storing instructions, the instructions configured to be executed by the one or more processors and cause the one or more processors to perform operations comprising: at an electronic device comprising a processor and memory storing instructions for execution by the processor:
providing, at the user device, a graphical user interface including an at least partially speech-based conversational interface for interacting with the user, the graphical user interface displaying at least a portion of a conversational interaction between the user and the user device;
obtaining context information associated with an interaction between the user and the user device;
receiving a speech input from the user through the conversational interface;
processing the speech input to determine a user intent associated with the speech input; and
upon determination that the user intent associated with the speech input is for invoking a software application installed on the user device:
invoking the software application on the user device external to the graphical user interface including the conversational interface; and
providing a response based on the user intent and the context information.
8. The system of claim 7 , wherein obtaining the context information further comprises:
receiving an additional speech input through the conversational interface prior to receiving the speech input; and
processing the additional speech input to obtain the context information.
9. The system of claim 7 , wherein processing the speech input to determine the user intent associated with the speech input further comprises:
disambiguating the speech input using the context information.
10. The system of claim 7 , wherein the operations further comprise:
receiving an additional speech input from the user through the conversational interface;
processing the additional speech input to determine an additional user intent associated with the additional speech input; and
based on the additional user intent that has been determined:
obtaining another input related to the additional user intent from the user through the conversational interface; and
executing a task for fulfilling the additional user intent within the conversational interface.
11. The system of claim 7 , wherein the operations further comprise:
receiving an additional speech input from the user through the conversational interface;
processing the additional speech input to determine an additional user intent associated with the additional speech input;
upon determination that the additional user intent associated with the additional speech input is for invoking an additional software application installed on the user device:
invoking the additional software application on the user device outside of the conversational interface.
12. The system of claim 7 , wherein obtaining the context information further comprises:
receiving non-speech input from the user prior to the speech input; and
processing the non-speech input to obtain the context information.
13. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device, cause the device to:
provide, at the user device, a graphical user interface including an at least partially speech-based conversational interface for interacting with the user, the graphical user interface displaying at least a portion of a conversational interaction between the user and the user device, the graphical user interface displaying at least a portion of a conversational interaction between the user and the user device;
obtain context information associated with an interaction between the user and the user device;
receive a speech input from the user through the conversational interface;
process the speech input to determine a user intent associated with the speech input; and
upon determination that the user intent associated with the speech input is for invoking a software application installed on the user device:
invoke the software application on the user device external to the graphical user interface including the conversational interface; and
provide a response based on the user intent and the context information.
14. The computer readable storage medium of claim 13 , wherein the instructions further cause the device to:
receive an additional speech input through the conversational interface prior to receiving the speech input; and
process the additional speech input to obtain the context information.
15. The computer readable storage medium of claim 13 , wherein processing the speech input to determine the user intent associated with the speech input further comprises:
disambiguating the speech input using the context information.
16. The computer readable storage medium of claim 13 , wherein the instructions further cause the device to:
receive an additional speech input from the user through the conversational interface;
process the additional speech input to determine an additional user intent associated with the additional speech input; and
based on the additional user intent that has been determined:
obtain another input related to the additional user intent from the user through the conversational interface; and
execute a task for fulfilling the additional user intent within the conversational interface.
17. The computer readable storage medium of claim 13 , wherein the instructions further cause the device to:
receive an additional speech input from the user through the conversational interface;
process the additional speech input to determine an additional user intent associated with the additional speech input;
upon determination that the additional user intent associated with the additional speech input is for invoking an additional software application installed on the user device:
invoke the additional software application on the user device outside of the conversational interface.
18. The computer readable storage medium of claim 13 , wherein obtaining the context information further comprises:
receiving non-speech input from the user prior to the speech input; and
processing the non-speech input to obtain the context information.
19. The method of claim 1 , wherein the speech input comprises a command to invoke the software application installed on the user device.
20. The system of claim 7 , wherein the speech input comprises a command to invoke the software application installed on the user device.
21. The computer readable storage medium of claim 13 , wherein the speech input comprises a command to invoke the software application installed on the user device.
22. The method of claim 1 , wherein displaying at least a portion of the conversational interaction includes displaying a paraphrase of user input.
23. The system of claim 7 , wherein displaying at least a portion of the conversational interaction includes displaying a paraphrase of user input.
24. The computer readable storage medium of claim 13 , wherein displaying at least a portion of the conversational interaction includes displaying a paraphrase of user input.