Initializing a conversation with an automated agent via selectable graphical element
Methods, apparatus, systems, and computer-readable media are provided for using selectable elements to invoke an automated assistant at a computing device. While operating the computing device, a user may not be aware that the automated assistant can be invoked according to certain invocation phrases. In order to inform the user of the functionality of the automated assistant, the user can be presented with selectable elements that can initialize the automated assistant when selected. Furthermore, a selectable element can provide an invocation phrase in textual form so that the user is aware of their ability to invoke the automated assistant by speaking the invocation phrase. The selectable element can be presented at different devices associated with the user, and the automated assistant can be initialized at a device that is separate from the device where the selectable element is presented.
1. A computer-implemented method, comprising:
receiving, by a server device, an indication of a user selection of a selectable element rendered at a graphical user interface of a first computing device,
wherein the selectable element is rendered at the graphical user interface of the first computing device while an application is executing at the first computing device, and
wherein the selectable element is rendered at the graphical user interface with a command phrase that includes natural language content, which, when provided as a spoken input by a user to an automated assistant interface, prior to the selectable element being rendered at the graphical user interface of the first computing device, transitions an automated assistant into an operating state in which the automated assistant causes an operation associated with the application to be performed; and
in response to receiving the indication of the user selection of the selectable element:
identifying multiple computing devices associated with an account of the user associated with the first computing device, wherein the multiple computing devices include at least the first computing device and one or more additional computing devices;
selecting a particular computing device of the one or more additional computing devices for initializing the automated assistant, from among the multiple computing devices, based on the operation associated with the application that is to be performed; and
causing, in response to the selection of the selectable element and based on selecting the particular computing device, the automated assistant to:
initialize at the automated assistant interface of the particular computing device,
perform the operation associated with the application,
transition into a state in which the automated assistant monitors, at the automated assistant interface of the particular computing device, for one or more additional spoken utterances of the user at the automated assistant interface of the particular computing device, and
provide one or more additional selectable elements at the automated assistant interface of the particular computing device, the one or more additional selectable elements being selectable by the user to cause the automated assistant at the automated assistant interface of the particular computing device to perform one or more additional operations associated with the application.
2. The computer-implemented method of claim 1 , further comprising:
providing, to the server device, credentials of the user; and
receiving, from the server device and based on the credentials of the user, a directory of computing devices associated with the user, wherein the directory identifies the one or more additional computing devices.
3. The computer-implemented method of claim 1 , further comprising: causing the first computing device or the particular computing device to audibly render the command phrase.
4. The computer-implemented method of claim 1 , wherein the command phrase has not been previously submitted by user via any spoken input to the automated assistant.
5. The computer-implemented method of claim 4 , further comprising:
providing, to the server device, data that identifies application content previously displayed at the graphical user interface while the application is executing and prior to rendering the command phrase with the selectable element, wherein at least a portion of the command phrase describes the previously displayed application content.
6. The computer-implemented method of claim 1 , wherein the server device causes the selectable element to be rendered based on data associated with an account profile of the user.
7. The computer-implemented method of claim 6 , wherein the operation associated with the application is performed using a particular value for a parameter, and wherein the server device determines the particular value for the parameter based on a frequency that the user has previously used the application to perform the operation using the particular value for the parameter.
8. A system comprising:
memory and one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the one or more processors to:
receive, by a server device, an indication of a user selection of a selectable element rendered at a graphical user interface of a first computing device,
wherein the selectable element is rendered at the graphical user interface of the first computing device while an application is executing at the first computing device, and
wherein the selectable element is rendered at the graphical user interface with a command phrase that includes natural language content, which, when provided as a spoken input by a user to an automated assistant interface, prior to the selectable element being rendered at the graphical user interface of the first computing device, transitions an automated assistant into an operating state in which the automated assistant causes an operation associated with the application to be performed; and
in response to receiving the indication of the user selection of the selectable element:
identify multiple computing devices associated with an account of the user associated with the first computing device, wherein the multiple computing devices include at least the first computing device and one or more additional computing devices;
select a particular computing device of the one or more additional computing devices for initializing the automated assistant, from among the multiple computing devices, based on the operation associated with the application that is to be performed; and
cause, in response to the selection of the selectable element and based on selecting the particular computing device, the automated assistant to:
initialize at the automated assistant interface of the particular computing device,
perform the operation associated with the application,
transition into a state in which the automated assistant monitors, at the automated assistant interface of the particular computing device, for one or more additional spoken utterances of the user at the automated assistant interface of the particular computing device, and
provide one or more additional selectable elements at the automated assistant interface of the particular computing device, the one or more additional selectable elements being selectable by the user to cause the automated assistant at the automated assistant interface of the particular computing device to perform one or more additional operations associated with the application.
9. The system of claim 8 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
provide, to the server device, credentials of the user; and
receive, from the server device and based on the credentials of the user, a directory of computing devices associated with the user, wherein the directory identifies the one or more additional computing devices.
10. The system of claim 8 , further comprising: causing the first computing device or the particular computing device to audibly render the command phrase.
11. The system of claim 8 , wherein the command phrase has not been previously submitted by user via any spoken input to the automated assistant.
12. The system of claim 11 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to: provide, to the server device, data that identifies application content previously displayed at the graphical user interface while the application is executing and prior to rendering the command phrase with the selectable element, wherein at least a portion of the command phrase describes the previously displayed application content.
13. The system of claim 8 , wherein the server device causes the selectable element to be rendered based on data associated with an account profile of the user.
14. The system of claim 13 , wherein the operation associated with the application is performed using a particular value for a parameter, and wherein the server device determines the particular value for the parameter based on a frequency that the user has previously used the application to perform the operation using the particular value for the parameter.
15. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
receive, by a server device, an indication of a user selection of a selectable element rendered at a graphical user interface of a first computing device,
wherein the selectable element is rendered at the graphical user interface of the first computing device while an application is executing at the first computing device, and
wherein the selectable element is rendered at the graphical user interface with a command phrase that includes natural language content, which, when provided as a spoken input by a user to an automated assistant interface, prior to the selectable element being rendered at the graphical user interface of the first computing device, transitions an automated assistant into an operating state in which the automated assistant causes an operation associated with the application to be performed; and
in response to receiving the indication of the user selection of the selectable element:
identify multiple computing devices associated with an account of the user associated with the first computing device, wherein the multiple computing devices include at least the first computing device and one or more additional computing devices;
select a particular computing device of the one or more additional computing devices for initializing the automated assistant, from among the multiple computing devices, based on the operation associated with the application that is to be performed; and
cause, in response to the selection of the selectable element and based on selecting the particular computing device, the automated assistant to:
initialize at the automated assistant interface of the particular computing device,
perform the operation associated with the application,
transition into a state in which the automated assistant monitors, at the automated assistant interface of the particular computing device, for one or more additional spoken utterances of the user at the automated assistant interface of the particular computing device, and
provide one or more additional selectable elements at the automated assistant interface of the particular computing device, the one or more additional selectable elements being selectable by the user to cause the automated assistant at the automated assistant interface of the particular computing device to perform one or more additional operations associated with the application.
16. The at least one non-transitory computer-readable medium of claim 15 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
provide, to the server device, credentials of the user; and
receive, from the server device and based on the credentials of the user, a directory of computing devices associated with the user, wherein the directory identifies the one or more additional computing devices.
17. The at least one non-transitory computer-readable medium of claim 15 , further comprising: causing the first computing device or the particular computing device to audibly render the command phrase.
18. The at least one non-transitory computer-readable medium of claim 15 , wherein the command phrase has not been previously submitted by user via any spoken input to the automated assistant.
19. The at least one non-transitory computer-readable medium of claim 18 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
provide, to the server device, data that identifies application content previously displayed at the graphical user interface while the application is executing and prior to rendering the command phrase with the selectable element, wherein at least a portion of the command phrase describes the previously displayed application content.
20. The at least one non-transitory computer-readable medium of claim 15 , wherein the server device causes the selectable element to be rendered based on data associated with an account profile of the user.