Intelligent automated assistant
An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.
1. An automated assistant operating on one or more computing devices, the automated assistant comprising:
an input device, for receiving user input;
a language interpreter component, for interpreting the received user input to derive a representation of user intent;
a dialog flow processor component, for identifying at least one task based at least in part on the derived representation of user intent;
a services orchestration component, for calling at least one service for performing the identified task; and
an output processor component, for causing a first output to be displayed prior to receiving the user input, and for causing a second output to be displayed based on data received from the at least one called service;
wherein the first output comprises a plurality of core competencies of the automated assistant and an example of a natural language input for invoking each of the plurality of core competencies.
2. A method for implementing an automated assistant on one or more computing devices having one or more processors and memory, the method comprising:
at the one or more computing devices:
invoking the automated assistant;
causing a first output to be displayed, wherein the first output comprises a plurality of core competencies of the automated assistant and an example of a natural language input for invoking each of the plurality of core competencies;
at an input device, receiving user input;
interpreting the received user input to derive a representation of user intent;
identifying at least one task based at least in part on the derived representation of user intent;
calling at least one service for performing the identified task; and
causing a second output to be displayed based on data received from the at least one called service;
wherein the first output is displayed prior to receiving the user input.
3. A non-transitory computer-readable medium for implementing an automated assistant on one or more computing devices, the computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:
invoking the automated assistant;
causing a first output to be displayed in a conversation interface of the automated assistant, wherein the first output comprises a plurality of core competencies of the automated assistant and an example of a natural language input for invoking each of the plurality of core competencies;
at an input device, receiving user input;
interpreting the received user input to derive a representation of user intent;
identifying at least one task based at least in part on the derived representation of user intent;
calling at least one service for performing the identified task; and
causing a second output to be displayed based on data received from the at least one called service;
wherein the first output is displayed prior to receiving the user input.
4. The automated assistant of claim 1 , wherein one of the plurality of core competencies is setting a reminder.
5. The automated assistant of claim 1 , wherein one of the plurality of core competencies is getting directions.
6. The automated assistant of claim 1 , wherein one of the plurality of core competencies is searching for a restaurant.
7. The automated assistant of claim 1 , wherein one of the plurality of core competencies is searching for an entertainment event.
8. The automated assistant of claim 1 , wherein one of the plurality of core competencies is getting a weather forecast.
9. The automated assistant of claim 1 , wherein one of the plurality of core competencies is booking a taxi.
10. The automated assistant of claim 1 , wherein the first output further comprises a graphical icon for each of the core competencies.
11. The automated assistant of claim 1 , wherein:
the first output further comprises a user interface element for exploring a core competency; and
in response to a selection of the user interface element, the output processor component causes a third output to be displayed comprising one or more additional examples of natural language input for invoking the core competency.
12. The automated assistant of claim 1 , wherein:
the input device receives a natural language user input from a user; and
in response to the natural language user input, the output processor component causes a third output to be displayed comprising the natural language user input and at least one of the plurality of core competencies.
13. The automated assistant of claim 12 , wherein:
the output processor component causes a fourth output to be displayed comprising a natural language response to the natural language user input and the at least one of the plurality of core competencies.
14. The automated assistant of claim 1 , wherein:
the input device receives a natural language user input from a user, the natural language user input requests a search for an event; and
the output processor component causes a third output to be displayed comprising an echo of the natural language user input, a paraphrase of the natural language user input, and at least one search result.
15. The automated assistant of claim 1 , wherein:
the input device receives a natural language user input from a user, the natural language user input specifying parameters for making a restaurant reservation;
in response to the natural language input, the output processor component causes a third output to be displayed comprising a summary of the parameters and a plurality of restaurants meeting the specified parameters; and
for at least one of the plurality of restaurants, the third output comprises displays a reservation option meeting the specified parameters and at least one additional reservation option different from the specified parameters.
16. The automated assistant of claim 15 , wherein the reservation option meeting the specified parameters includes a reservation for a specified time in the natural language input, and the additional reservation option different from the specified parameters includes a reservation option for a different time from the specified time.
17. The automated assistant of claim 1 , wherein:
the input device receives a natural language question from a user; and
the output processor component causes a third output to be displayed comprising an echo of the natural language question, a natural language answer to the natural language question, and an excerpt retrieved from an information source based on which the natural language answer is produced.
18. The automated assistant of claim 17 , wherein the natural language question is related to weather, the natural language answer describes the weather in a natural language format, and the excerpt is a weather forecast excerpt.
19. The method of claim 2 , wherein one of the plurality of core competencies is setting a reminder.
20. The method of claim 2 , wherein one of the plurality of core competencies is getting directions.
21. The method of claim 2 , wherein one of the plurality of core competencies is searching for a restaurant.
22. The method of claim 2 , wherein one of the plurality of core competencies is searching for an entertainment event.
23. The method of claim 2 , wherein one of the plurality of core competencies is getting a weather forecast.
24. The method of claim 2 , wherein one of the plurality of core competencies is booking a taxi.
25. The method of claim 2 , wherein the first output further comprises a graphical icon for each of the core competencies.
26. The method of claim 2 , wherein the first output further comprises a
user interface element for exploring a core competency; and
in response to a selection of the user interface element, causing to be displayed one or more additional examples of natural language input for invoking the core competency.
27. The method of claim 2 , further comprising:
at the input device, receiving a natural language user input from a user; and
in response to the natural language user input, presenting the natural language user input concurrently with at least one of the plurality of core competencies.
28. The method of claim 27 , further comprising:
presenting a natural language response to the natural language user input concurrently with the at least one of the plurality of core competencies.
29. The method of claim 2 , further comprising:
at the input device, receiving a natural language user input from a user, the natural language user input requests a search for an event; and
concurrently presenting an echo of the natural language user input, a paraphrase of the natural language user input, and at least one search result.
30. The method of claim 2 , further comprising:
at the input device, receiving a natural language user input from a user, the natural language user input specifying parameters for making a restaurant reservation;
in response to the natural language input, concurrently presenting a summary of the parameters, and a plurality of restaurants meeting the specified parameters; and
for at least one of the plurality of restaurants, displaying a reservation option meeting the specified parameters, and at least one additional reservation option different from the specified parameters.
31. The method of claim 30 , wherein the reservation option meeting the specified parameters includes a reservation for a specified time in the natural language input, and the additional reservation option different from the specified parameters includes a reservation option for a different time from the specified time.
32. The method of claim 2 , further comprising:
at the input device, receiving a natural language question from a user; and
concurrently displaying an echo of the natural language question, a natural language answer to the natural language question, and an excerpt retrieved from an information source based on which the natural language answer is produced.
33. The method of claim 32 , wherein the natural language question is related to weather, the natural language answer describes the weather in a natural language format, and the excerpt is a weather forecast excerpt.
34. The computer-readable medium of claim 3 , wherein one of the plurality of core competencies is setting a reminder.
35. The computer-readable medium of claim 3 , wherein one of the plurality of core competencies is getting directions.
36. The computer-readable medium of claim 3 , wherein one of the plurality of core competencies is searching for a restaurant.
37. The computer-readable medium of claim 3 , wherein one of the plurality of core competencies is searching for an entertainment event.
38. The computer-readable medium of claim 3 , wherein one of the plurality of core competencies is getting a weather forecast.
39. The computer-readable medium of claim 3 , wherein one of the plurality of core competencies is booking a taxi.
40. The computer-readable medium of claim 3 , wherein the first output further comprises a graphical icon for each of the respective core competencies.
41. The computer-readable medium of claim 3 , wherein the first output further comprises a
user interface element for exploring a core competency; and
wherein the operations further comprise:
in response to a selection of the user interface element, causing to be displayed one or more additional examples of natural language input for invoking the core competency.
42. The computer-readable medium of claim 3 , wherein the operations further comprise:
receiving a natural language user input from a user; and
in response to the natural language user input, presenting the natural language user input concurrently with at least one of the plurality of core competences.
43. The computer-readable medium of claim 42 , wherein the operations further comprise:
presenting a natural language response to the natural language user input concurrently with the at least one of the plurality of core competencies.
44. The computer-readable medium of claim 3 , wherein the operations further comprise:
receiving a natural language user input from a user, the natural language user input requests a search for an event; and
concurrently presenting an echo of the natural language user input, a paraphrase of the natural language user input, and at least one search result.
45. The computer-readable medium of claim 3 , wherein the operations further comprise:
receiving a natural language user input from a user, the natural language user input specifying parameters for making a restaurant reservation;
in response to the natural language input, concurrently presenting a summary of the parameters, and a plurality of restaurants meeting the specified parameters; and
for at least one of the plurality of restaurants, displaying a reservation option meeting the specified parameters, and at least one additional reservation option different from the specified parameters.
46. The computer-readable medium of claim 45 , wherein the reservation option meeting the specified parameters includes a reservation for a specified time in the natural language input, and the additional reservation option different from the specified parameters includes a reservation option for a different time from the specified time.
47. The computer-readable medium of claim 3 , wherein the operations further comprise:
receiving a natural language question from a user; and
concurrently displaying an echo of the natural language question, a natural language answer to the natural language question, and an excerpt retrieved from an information source based on which the natural language answer is produced.
48. The computer-readable medium of claim 47 , wherein the natural language question is related to weather, the natural language answer describes the weather in a natural language format, and the excerpt is a weather forecast excerpt.