IP Library Granted Patent US 11,769,184
Granted Patent B2
US 11,769,184 · App. 18/094,256 · Granted Sep 26, 2023

Enhancing functionalities of virtual assistants and dialog systems via plugin marketplace

Inventors: Ilya Gennadyevich Gelfenbeyn (Sunnyvale, CA); Artem Goncharuk (Mountain View, CA); Pavel Aleksandrovich Sirotin (Sunnyvale, CA)
Assignee: GOOGLE LLC
G06Q30/0601
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,769,184
App. No.
18/094,256
Granted
Sep 26, 2023
Kind
B2
Abstract

A method for enhancing dialog systems is disclosed herein. The method may include maintaining an online marketplace that may have a plurality of dialog system extension elements. The plurality of dialog system extension elements may include at least one of a dialog system plugin, a dialog system add-on, a dialog system update, and a dialog system upgrade. The method may further include receiving a selection of one of the plurality of dialog system extension elements from an end user. The end user may be associated with a dialog system. The method may continue with associating the one of the plurality of dialog system extension elements with the dialog system of the end user.

Claims (72)

1. A method performed by at least one processor, the method comprising:

maintaining a plurality of dialog system extension elements, wherein each of the plurality of dialog system extension elements includes a dialog system plugin, a dialog system add-on, a dialog system update, or a dialog system upgrade;

receiving a speech-based user request from an end user via a dialog system interface of a dialog system associated with the end user, wherein the dialog system is associated with a virtual assistant at least partially installed on a user device of the end user, wherein the dialog system interface is running on the user device, and wherein one or more existing dialog system extension elements are already installed in the dialog system;

processing the user request using at least an automatic speech recognizer and a natural language processing module;

identifying, from the plurality of dialog extension elements, a given dialog system extension element, of the plurality of dialog system extension elements, that is capable of processing the user request, wherein the given dialog extension element is in addition to the existing dialog extension elements;

determining that the dialog system associated with the end user cannot process the user request with functionalities of the one or more existing dialog system extension elements of the dialog system that is associated with the end user;

in response to determining that the dialog system associated with the end user cannot process the user request:

providing a recommendation to the end user to authorize installing the given dialog system extension element in the dialog system;

subsequent to providing the recommendation to the end user to authorize installing the given dialog system extension element in the dialog system:

receiving, responsive to a selection from the end user, a request for installing the given dialog system extension element in the dialog system; and

installing, based on the received request, the given dialog system extension element in the dialog system.

2. The method of claim 1 , further comprising:

subsequent to installing the given dialog extension element in the dialog system:

receiving a speech-based additional user request from the end user via the dialog system interface of the dialog system associated with the end user;

identifying that the given dialog system extension element is applicable for processing the additional user request; and

in response to identifying that the given dialog system extension element is applicable for processing the additional user request, and based on the given dialog system extension element being installed:

causing the additional user request to be processed with the given dialog system extension element.

3. The method of claim 2 , further comprising:

causing an audio response, from the processing with the given dialog system extension element, to be provided via the dialog system interface in response to the additional user request.

4. The method of claim 3 , further comprising:

causing the additional user request to also be processed, by the dialog engine independent of the given dialog system extension element, to generate an additional audio response; and

causing the additional audio response to be provided via the dialog system interface, along with the audio response and in response to the additional user request.

5. The method of claim 2 , further comprising:

selecting the given dialog extension element, for use in the recommendation, based on the given dialog extension element being designed to handle a type of user requests, and based on the processing indicating that the user request is of the type.

6. The method of claim 1 , wherein the given dialog system extension element is created by a developer and provided, by the developer, for inclusion among the plurality of dialog extension elements.

7. The method of claim 1 , wherein the automatic speech recognizer is a dynamically created recognizer.

8. The method of claim 1 , wherein the dialog system interface is implemented as a firmware application.

9. The method of claim 1 , further comprising:

selecting the given dialog extension element, for use in the recommendation, based on the given dialog extension element being designed to handle a type of user requests, and based on the processing indicating that the user request is of the type.

10. The method of claim 1 , further comprising:

selecting the given dialog extension element, for use in the recommendation, based on a ranking of the given dialog extension element among the plurality of dialog extension elements.

11. The method of claim 1 , wherein installing the given dialog system extension element in the dialog system comprises:

responsive to receiving the authorization, using associated authorization data to link the given dialog extension element with the dialog system associated with the end user.

12. The method of claim 11 , wherein using the associated authorization data to link the given dialog extension element with the dialog system associated with the end user comprises embedding the given dialog extension element into the dialog system associated with the end user.

13. The method of claim 1 , wherein the given dialog extension element is a dialog system plugin.

14. A system, comprising:

memory operable to maintain a plurality of dialog system extension elements, wherein each of the plurality of dialog system extension elements includes a dialog system plugin, a dialog system add-on, a dialog system update, or a dialog system upgrade; and

at least one processor operable to:

receive a speech-based user request from an end user via a dialog system interface of a dialog system associated with the end user, wherein the dialog system is associated with a virtual assistant at least partially installed on a user device of the end user, wherein the dialog system interface is running on the user device, and wherein one or more existing dialog system extension elements are already installed in the dialog system;

process the user request using at least an automatic speech recognizer and a natural language processing module;

identify, from the plurality of dialog extension elements, a given dialog system extension element, of the plurality of dialog system extension elements, that is capable of processing the user request, wherein the given dialog extension element is in addition to the existing dialog extension elements;

determine that the dialog system associated with the end user cannot process the user request with functionalities of the one or more existing dialog system extension elements of the dialog system that is associated with the end user;

in response to determining that the dialog system associated with the end user cannot process the user request:

provide a recommendation to the end user to authorize installing the given dialog system extension element in the dialog system;

subsequent to providing the recommendation to the end user to authorize installing the given dialog system extension element in the dialog system:

receive, responsive to a selection from the end user, a request for installing the given dialog system extension element in the dialog system; and

install, based on the received request, the given dialog system extension element in the dialog system.

15. The system of claim 14 , wherein the at least one processor is further operable to:

subsequent to installing the given dialog extension element in the dialog system:

receive a speech-based additional user request from the end user via the dialog system interface of the dialog system associated with the end user;

identify that the given dialog system extension element is applicable for processing the additional user request; and

in response to identifying that the given dialog system extension element is applicable for processing the additional user request, and based on the given dialog system extension element being installed:

cause the additional user request to be processed with the given dialog system extension element.

16. The system of claim 15 , wherein the at least one processor is further operable to:

cause an audio response, from the processing with the given dialog system extension element, to be provided via the dialog system interface in response to the additional user request.

17. The system of claim 16 , wherein the at least one processor is further operable to:

causing the additional user request to also be processed, by the dialog engine independent of the given dialog system extension element, to generate an additional audio response; and

causing the additional audio response to be provided via the dialog system interface, along with the audio response and in response to the additional user request.

18. The system of claim 16 , wherein the dialog system interface is implemented as a firmware application.

19. The system of claim 16 , wherein the at least one processor is further operable to:

select the given dialog extension element, for use in the recommendation, based on the given dialog extension element being designed to handle a type of user requests, and based on the processing indicating that the user request is of the type.

20. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

maintaining a plurality of dialog system extension elements, wherein each of the plurality of dialog system extension elements includes a dialog system plugin, a dialog system add-on, a dialog system update, or a dialog system upgrade;

receiving a speech-based user request from an end user via a dialog system interface of a dialog system associated with the end user, wherein the dialog system is associated with a virtual assistant at least partially installed on a user device of the end user, wherein the dialog system interface is running on the user device, and wherein one or more existing dialog system extension elements are already installed in the dialog system;

processing the user request using at least an automatic speech recognizer and a natural language processing module;

identifying, from the plurality of dialog extension elements, a given dialog system extension element, of the plurality of dialog system extension elements, that is capable of processing the user request, wherein the given dialog extension element is in addition to the existing dialog extension elements;

determining that the dialog system associated with the end user cannot process the user request with functionalities of the one or more existing dialog system extension elements of the dialog system that is associated with the end user;

in response to determining that the dialog system associated with the end user cannot process the user request:

providing a recommendation to the end user to authorize installing the given dialog system extension element in the dialog system;

subsequent to providing the recommendation to the end user to authorize installing the given dialog system extension element in the dialog system:

receiving, responsive to a selection from the end user, a request for installing the given dialog system extension element in the dialog system; and

installing, based on the received request, the given dialog system extension element in the dialog system.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 6, 2023
From: GELFENBEYN, ILYA GENNADYEVICH; GONCHARUK, ARTEM; SIROTIN, PAVEL ALEKSANDROVICH
To: SPEAKTOIT, INC.
Reel/Frame 062604/0894 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 6, 2023
From: SPEAKTOIT, INC.
To: GOOGLE INC.
Reel/Frame 062604/0902 →
CHANGE OF NAME Recorded Feb 6, 2023
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 062683/0164 →
Continuity (5)
Continuation 17521115 · Nov 8, 2021
Continuation 16424783 · May 29, 2019
Continuation 15166293 · May 27, 2016
Provisional Application 62166686 · May 27, 2015
Related Publication 20230153876A1 · May 18, 2023