Granted Patent B2

US 12,271,934 · App. 18/372,483 · Granted Apr 8, 2025

Inventors: Ilya Gennadyevich Gelfenbeyn (Sunnyvale, CA); Artem Goncharuk (Mountain View, CA); Pavel Aleksandrovich Sirotin (Sunnyvale, CA)

Assignee: GOOGLE LLC

G06Q30/0601

Abstract

Claims (63)

1. A method performed by one or more processors, the method comprising:

maintaining a plurality of plugins each created by a corresponding developer, wherein the plugins include a given plugin that is created by a given developer of the corresponding developers;

receiving, from an end user, a selection of the given plugin from the plurality of maintained plugins;

in response to receiving the selection:

linking the given plugin with a dialog system that is associated with the end user, wherein the dialog system is associated with a virtual assistant at least partially installed on a user device of the end user;

subsequent to linking the given plugin with the dialog system that is associated with the end user:

receiving a user request from the end user via a dialog system interface of the dialog system that is associated with the end user, wherein the dialog system interface is running on the user device;

identifying the given plugin based on the linking with the dialog system and based on the user request being received via the dialog interface of the dialog system;

processing the user request using the dialog system and without using the given plugin;

determining, based on processing the user request using the dialog system interface and without using the given plugin, to use the given plugin in processing the user request,

wherein determining to use the given plugin is further based on the linking with the dialog system and based on the user request being received via the dialog interface of the dialog system; and

in response to determining to use the dialog extension element in processing the user request:

using the dialog extension element in generating a response to the user request; and

causing the response to be rendered at the user device via the dialog system and responsive to the user request.

2. The method of claim 1 , wherein the user request is a speech-based user request and further comprising:

processing the speech-based user request using an automatic speech recognizer to generate recognized text;

wherein processing the user request using the dialog system and without using the dialog extension element comprises processing the recognized text using the dialog system.

3. The method of claim 2 , wherein the automatic speech recognizer is a dynamically created recognizer.

4. The method of claim 1 , further comprising:

subsequent to linking the given plugin with the dialog system that is associated with the end user:

receiving an additional user request from the end user via the dialog system interface;

identifying the given plugin based on the linking with the dialog system and based on the user request being received via the dialog interface of the dialog system;

processing the additional user request using the dialog system and without using the given plugin;

determining, based on processing the additional user request using the dialog system interface and without using the given plugin, to provide an additional response, to the additional user request, without using the given plugin in processing the additional user request; and

causing the additional response to be rendered at the user device via the dialog system and responsive to the additional user request.

5. The method of claim 1 , wherein the response is an audio response.

6. The method of claim 1 , wherein the dialog system interface is implemented as a mobile application.

7. The method of claim 1 , wherein linking the given plugin with the dialog system comprises:

using authorization data to link the given plugin with the dialog system associated with the end user.

8. The method of claim 7 , wherein the authorization data is for an online marketplace at which the plugins are maintained.

9. The method of claim 1 , wherein the given plugin is associated with metadata that includes a description of the plugin.

10. A system, comprising:

memory operable to maintain a plurality of plugins each created by a corresponding developer, wherein the plugins include a given plugin that is created by a given developer of the corresponding developers; and

one or more processors operable to:

receive, from an end user, a selection of the given plugin from the plurality of maintained plugins;

in response to receiving the selection:

link the given plugin with a dialog system that is associated with the end user, wherein the dialog system is associated with a virtual assistant at least partially installed on a user device of the end user;

subsequent to linking the given plugin with the dialog system that is associated with the end user:

receive a user request from the end user via a dialog system interface of the dialog system that is associated with the end user, wherein the dialog system interface is running on the user device;

identify the given plugin based on the linking with the dialog system and based on the user request being received via the dialog interface of the dialog system;

process the user request using the dialog system and without using the given plugin;

determine, based on processing the user request using the dialog system interface and without using the given plugin, to use the given plugin in processing the user request,

wherein in determining to use the given plugin one or more of the processors are to determine to use the plugin further based on the linking with the dialog system and based on the user request being received via the dialog interface of the dialog system; and

in response to determining to use the dialog extension element in processing the user request:

use the dialog extension element in generating a response to the user request; and

cause the response to be rendered at the user device via the dialog system and responsive to the user request.

11. The system of claim 10 , wherein the user request is a speech-based user request and one or more of the processors are further operable to:

process the speech-based user request using an automatic speech recognizer to generate recognized text;

wherein in processing the user request using the dialog system and without using the dialog extension element one or more of the processors are to process the recognized text using the dialog system.

12. The system of claim 11 , wherein the automatic speech recognizer is a dynamically created recognizer.

13. The system of claim 11 , one or more of the processors are further operable to:

subsequent to linking the given plugin with the dialog system that is associated with the end user:

receive an additional user request from the end user via the dialog system interface;

identify the given plugin based on the linking with the dialog system and based on the user request being received via the dialog interface of the dialog system;

process the additional user request using the dialog system and without using the given plugin;

determine, based on processing the additional user request using the dialog system interface and without using the given plugin, to provide an additional response, to the additional user request, without using the given plugin in processing the additional user request; and

cause the additional response to be rendered at the user device via the dialog system and responsive to the additional user request.

14. The system of claim 11 , wherein the response is an audio response.

15. The system of claim 11 , wherein the dialog system interface is implemented as a mobile application.

16. The system of claim 11 , wherein in linking the given plugin with the dialog system one or more of the processors are to:

use authorization data to link the given plugin with the dialog system associated with the end user.

17. The system of claim 16 , wherein the authorization data is for an online marketplace at which the plugins are maintained.

18. The system of claim 16 , wherein the given plugin is associated with metadata that includes a description of the plugin.

Assignments (3)

Continuity (6)

References Cited (65)