Component libraries for voice interaction services
The disclosed embodiments include computerized methods, systems, and devices, including computer programs encoded on a computer storage medium, for integrating voice-based interaction and control into a native graphical user interface (GUI) of an executed application. For example, a communications device may obtaining component data identifying a plurality of components of a voice-user interface from a computing system maintained by a voice-service provider, and may execute an application linked to a corresponding one of the components of the voice-user interface. The communications device may generate the native GUI based on an output of the executed application, and may generate an interface element representative of the corresponding one of the components of the voice-user interface. The communications device may present the generated interface element within the native GUI, which may embed the corresponding component of the voice-user interface into the native GUI.
1. A computer-implemented method, comprising:
receiving, by a computing device and with a first client application installed on the computing device, an utterance that was spoken by a user of the computing device;
processing the utterance with the computing device to generate a voice query, the voice query including data that is based on the utterance;
transmitting the voice query from the computing device to a remote computing system corresponding to a voice service provider application installed on the computing device, wherein the voice service provider application is separate from the first client application;
obtaining, by the computing device and from the remote computing system associated with the voice service provider application, a response to the voice query;
selecting, by the first client application, a user interface element from a library of user interface elements associated with the voice service provider application; and
presenting the response to the voice query at the computing device,
wherein the user interface element from the library associated with the voice service provider is embedded into a native user interface of the first client application,
wherein the library associated with the voice service provider application comprises a library of application-neutral elements of executable code that, when executed, embed user interface elements into native user interfaces of a plurality of applications executed by the computing device, and
wherein the embedded user interface elements are selectable to initiate, by the voice service provider application, voice-based interactions with the native user interfaces of the plurality of applications executed by the computing device.
2. The computer-implemented method of claim 1 , wherein the response to the voice query comprises a structured data set that specifies values for a plurality of query-response parameters.
3. The computer-implemented method of claim 2 , wherein presenting the response to the voice query at the computing device further comprises:
parsing the structured data set of the response to the voice query to extract values for at least a portion of the plurality of query-response parameters; and
populating the user interface element with extracted values for the at least the portion of the plurality of query-response parameters.
4. The computer-implemented method of claim 1 , wherein selecting the user interface element from the library associated with the voice service provider application comprises selecting the user interface element from a plurality of user interface elements based on the selected user interface element corresponding to a type of inquiry provided by the utterance.
5. The computer-implemented method of claim 4 , wherein the plurality of user interface elements include at least one user interface element that is formatted to present responses to at least one of a generic inquiry, a weather inquiry, or a current events inquiry.
6. The computer-implemented method of claim 1 , wherein presenting the response to the voice query includes determining, by the computing device, a position in which to display the user interface element from the library associated with the voice service provider within the native user interface of the first client application.
7. The computer-implemented method of claim 1 , wherein presenting the response to the voice query includes configuring the user interface element from the library associated with the voice service provider as an overlay that obscures a portion of the native user interface of the first client application, a slide-up card or slide-down card that translates into or out of the native user interface of the first client application along a longitudinal axis, or a drawer card that translates into or out of the native user interface of the first client application along a transverse axis.
8. The computer-implemented method of claim 1 , wherein obtaining the response to the voice query includes obtaining (i) a speech component of the response to the voice query and (ii) a visual component of the response to the voice query,
wherein presenting the response to the voice query at the computing device comprises playing the speech component of the response to the voice query and displaying the visual component of the response to the voice query.
9. The computer-implemented method of claim 1 , wherein the response to the voice query includes weather information.
10. The computer-implemented method of claim 1 , wherein a developer of the first client application is independent of the remote service provider.
11. One or more non-transitory computer-readable media having instructions stored thereon that, when executed by data processing apparatus of a computing device, cause the data processing apparatus to perform operations comprising:
receiving, by the computing device and with a first client application installed on the computing device, an utterance that was spoken by a user of the computing device;
processing the utterance with the computing device to generate a voice query, the voice query including data that is based on the utterance;
transmitting the voice query from the computing device to a remote computing system associated with a voice service provider application installed on the computing device, wherein the voice service provider application is separate from the first client application;
obtaining, by the computing device and from the remote computing system associated with the voice service provider application, a response to the voice query;
selecting, by the first client application, a user interface element from a library of user interface elements associated with the voice service provider application; and
presenting the response to the voice query at the computing device,
wherein the user interface element from the library associated with the voice service provider is embedded into a native user interface of the first client application,
wherein the library associated with the voice service provider application comprises a library of application-neutral elements of executable code that, when executed, embed user interface elements into native user interfaces of a plurality of applications executed by the computing device, and
wherein the embedded user interface elements are selectable to initiate, by the voice service provider application, voice-based interactions with the native user interfaces of the plurality of applications executed by the computing device.
12. The one or more non-transitory computer-readable media of claim 11 , wherein the response to the voice query comprises a structured data set that specifies values for a plurality of query-response parameters.
13. The one or more non-transitory computer-readable media of claim 12 , wherein presenting the response to the voice query at the computing device further comprises:
parsing the structured data set of the response to the voice query to extract values for at least a portion of the plurality of query-response parameters; and
populating the user interface element with extracted values for the at least the portion of the plurality of query-response parameters.
14. The one or more non-transitory computer-readable media of claim 11 , wherein selecting the user interface element from the library associated with the voice service provider application comprises selecting the user interface element from a plurality of user interface elements based on the selected user interface element corresponding to a type of inquiry provided by the utterance.
15. The one or more non-transitory computer-readable media of claim 14 , wherein the plurality of user interface elements include at least one user interface element that is formatted to present responses to at least one of a generic inquiry, a weather inquiry, or a current events inquiry.
16. The one or more non-transitory computer-readable media of claim 11 , wherein presenting the response to the voice query includes determining, by the computing device, a position in which to display the user interface element from the library associated with the voice service provider within the native user interface of the first client application.
17. The one or more non-transitory computer-readable media of claim 11 , wherein presenting the response to the voice query includes configuring the user interface element from the library associated with the voice service provider as an overlay that obscures a portion of the native user interface of the first client application, a slide-up card or slide-down card that translates into or out of the native user interface of the first client application along a longitudinal axis, or a drawer card that translates into or out of the native user interface of the first client application along a transverse axis.
18. The one or more non-transitory computer-readable media of claim 11 , wherein obtaining the response to the voice query includes obtaining (i) a speech component of the response to the voice query and (ii) a visual component of the response to the voice query,
wherein presenting the response to the voice query at the computing device comprises playing the speech component of the response to the voice query and displaying the visual component of the response to the voice query.
19. The one or more non-transitory computer-readable media of claim 11 , wherein the response to the voice query includes weather information.
20. The one or more non-transitory computer-readable media of claim 11 , wherein a developer of the first client application is independent of the remote service provider.