Using frames for action dialogs
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using frames for performing tasks. One of the methods includes receiving a first request to perform a task, the first request comprising user speech identifying the task; generating a frame associated with the task, wherein the frame comprises one or more types of values necessary to perform the task, and wherein each type of value can be satisfied by a respective value; receiving a second request to provide information related to a question, the second request comprising user speech identifying the question; providing information identifying the question to a search engine, and receiving a response identifying one or more terms; determining that at least one term can satisfy a type of value necessary to perform the task; and storing the at least one term in the frame.
1 . A method, comprising:
generating, based on user interface input provided at a user device, a first inquiry associated with a calendar application;
causing, based on the first inquiry, a frame to be generated specifying one or more types of values necessary to respond to the first inquiry;
generating, based on additional user interface input provided at the user device subsequent to receiving the first inquiry, a second inquiry associated with a request for particular information of the calendar application associated with the first inquiry;
causing, based on the second inquiry, generation of a plurality of results responsive to the request for the particular information;
determining, based on processing the plurality of results, that one or more of the plurality of results, that are caused to be generated based on the second inquiry, provide at least one value corresponding to one or more of the types of values necessary to respond to the first inquiry specified by the frame;
storing, in the frame associated with the first inquiry, the at least one value that was provided by the one or more of the plurality of results,
wherein storing the at least one value is responsive to determining that the one or more of the plurality of results responsive to the second inquiry, associated with the request for particular information of the calendar application associated with the first inquiry, provides the at least one value corresponding to the one or more types of values specified by the frame associated with the first inquiry; and
causing a response to the first inquiry to be rendered based on the at least one value stored in the frame.
2 . The method of claim 1 , further comprising:
identifying, prior to generating the second inquiry, an ambiguity associated with one or more terms of the additional user interface input;
causing a disambiguation question identifying the different meanings to be provided to the user;
receiving further user interface input identifying a particular meaning associated with the terms responsive to the disambiguation question; and
generating, based on the additional user interface input provided at the user device subsequent to receiving the first inquiry and the further user interface input, the second inquiry associated with the request for information of the calendar application.
3 . The method of claim 1 , wherein the first inquiry is associated with a time of the calendar application and the one or more types of values include at least one of: a separate time associated with the calendar application.
4 . The method of claim 1 , wherein the one or more types of values include at least one of: a calendar event associated with the calendar application.
5 . The method of claim 1 , further comprising:
identifying, prior to storing the at least one value that was provided by the one or more of the plurality of results, a natural language question template associated with the at least one value;
causing, based on the natural language question template, a natural language question to be provided to the user via the user device; and
storing, in the frame associated with the first inquiry and based further on another user interface input responsive to the natural language question, the at least one value.
6 . The method of claim 1 , further comprising:
determining, prior to storing the at least one value that was provided by the one or more of the plurality of results, that the one or more of the plurality of results provides at least one additional value corresponding to one or more of the types of values specified by the frame;
determining respective confidence levels for the at least one value and the at least one additional value based on one or more rules; and
storing, in the frame associated with the first inquiry, the at least one value based further on the respective confidence levels.
7 . The method of claim 1 , wherein causing a response to the first inquiry to be rendered based on the at least one value stored in the frame, further comprises:
generating a calendar event associated with the calendar application; and
rendering a prompt for the user to schedule the calendar event.
8 . The method of claim 1 , wherein causing a response to the first inquiry to be rendered based on the at least one value stored in the frame, further comprises:
generating a reminder associated with the calendar application; and
causing the user to be prompted to set the reminder.
9 . A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations, the operations comprising:
receiving a first inquiry, wherein the first inquiry comprises natural language input from a user identifying a calendar application associated with the first inquiry;
in response to receiving the first inquiry:
generating a frame associated with the first inquiry, wherein the frame specifies one or more types of values necessary to respond to the first inquiry; and
receiving, subsequent to receiving the first inquiry, a second inquiry comprising a request by the user for a search system to access user data of the calendar application associated with the first inquiry;
in response to receiving the second inquiry:
causing the search system to execute a search over the user data based on the second inquiry;
receiving, based on the search, a plurality of results from the search system; and
determining, that one or more of the plurality of results that are caused to be generated based on the second inquiry provide at least one value corresponding to one or more of the types of values necessary to respond to the first inquiry;
storing, in the frame associated with the first inquiry, the at least one value,
wherein storing the at least one value is responsive to determining that one or more of the plurality of results, determined in response to receiving the second inquiry comprising the request by the user for the search system to access user data of the calendar application associated with the first inquiry, provide the at least one value corresponding to one or more of the types of values necessary to respond to the first inquiry; and
causing a response to the first inquiry to be rendered based on the at least one value stored in the frame.
10 . The system of claim 9 , the operations further comprising:
identifying, prior to causing the search system to execute a search over the user data based on the second inquiry, an ambiguity associated with the second inquiry;
causing a disambiguation inquiry identifying ambiguous meanings of one or more terms of the second inquiry to be provided to the user;
receiving a disambiguation response identifying a particular meaning associated with the terms responsive to the disambiguation inquiry; and
causing the search system to execute a search over the user data based further on the disambiguation response.
11 . The system of claim 9 , wherein the one or more types of values include at least one of: a date, a time, a location, a phone number, a user name, a web address, and/or a person's name.
12 . The system of claim 9 , wherein the response to the first inquiry comprises: setting a reminder, scheduling a calendar event, providing information, and/or providing directions.
13 . The system of claim 9 , the operations further comprising:
identifying, prior to storing the at least one value that was provided by the one or more of the plurality of results, a question template for the at least one value;
causing the question template to be provided for presentation to the user via the user device; and
storing, in the frame associated with the first inquiry, the at least one value, based further on a response, received by the user, to the natural language question.
14 . The system of claim 9 , the operations further comprising:
determining, prior to storing the at least one value that was provided by the one or more of the plurality of results, that the one or more of the plurality of results provides at least one additional value corresponding to one or more of the types of values specified by the frame;
determining respective confidence levels for the at least one value and the at least one additional value based on one or more rules; and
storing, in the frame associated with the first inquiry, the at least one value based further on the respective confidence levels.
15 . The system of claim 9 , wherein the operations are locally performed at a client computer of the user having a graphical user interface.
16 . The system of claim 9 , wherein the operations are remotely performed at a server.
17 . One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations, the operations comprising:
generating, based on user interface input provided at a user device, a first inquiry associated with a calendar application;
causing, based on the first inquiry, a frame to be generated specifying one or more types of values necessary to respond to the first inquiry; and
generating, based on additional user interface input provided at the user device subsequent to receiving the first inquiry, a second inquiry associated with a request for information of the calendar application associated with the first inquiry;
causing, based on the second inquiry, generation of a plurality of results responsive to the request for information; and
determining, based on processing the plurality of results, that one or more of the plurality of results, that are caused to be generated based on the second inquiry, provide at least one value corresponding to one or more of the types of values necessary to respond to the first inquiry specified by the frame;
storing, in the frame associated with the first inquiry, the at least one value that was provided by the one or more of the plurality of results,
wherein storing the at least one value is responsive to determining that the one or more of the plurality of results responsive to the second inquiry provides the at least one value corresponding to the one or more types of values specified by the frame associated with the first inquiry; and
causing a response to the first inquiry to be rendered based on the at least one value stored in the frame.
18 . The one or more non-transitory computer-readable storage media of claim 17 , wherein the operations are locally performed at a client computer of the user having a graphical user interface.
19 . The one or more non-transitory computer-readable storage media of claim 17 , wherein the operations are remotely performed at a server.
20 . The one or more non-transitory computer-readable storage media of claim 17 , wherein a first set of the operations are locally performed at a client computer of the user having a graphical user interface, and a second set of the operations are remotely performed at a server.