IP Library Granted Patent US 11,562,742
Granted Patent B2
US 11,562,742 · App. 17/150,139 · Granted Jan 24, 2023

Generating and transmitting invocation request to appropriate third-party agent

Inventors: Ulas Kirazci (Mountain View, CA); Bo Wang (San Jose, CA); Steve Chen (Los Altos, CA); Sunil Vemuri (Pleasanton, CA); Barnaby James (Los Gatos, CA); Valerie Nygaard (San Francisco, CA)
Assignee: GOOGLE LLC
G10L15/22G06F3/167G06F16/9535G06F40/30G10L15/1815G10L15/1822G10L15/30G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,562,742
App. No.
17/150,139
Granted
Jan 24, 2023
Kind
B2
Abstract

Some implementations are directed to selective invocation of a particular third-party (3P) agent by an automated assistant to achieve an intended action determined by the automated assistant during a dynamic dialog between the automated assistant and a user. In some of those implementations, the particular 3P agent is invoked with value(s) for parameter(s) that are determined during the dynamic dialog; and/or the particular 3P agent is selected, from a plurality of candidate 3P agents, for invocation based on the determined value(s) for the parameter(s) and/or based on other criteria. In some of those implementations, the automated assistant invokes the particular 3P agent by transmitting, to the particular 3P agent, a 3P invocation request that includes the determined value(s) for the parameter(s).

Claims (81)

1. A method implemented by one or more processors, comprising:

receiving, over one or more network interfaces, user input transmitted by a client device of a user, the user input being free-form and being directed to an automated assistant interface implemented by the client device;

determining, based on processing the user input, that the user input indicates an intended action and indicates a value for a parameter for the intended action;

selecting, based on the intended action and the value of the parameter, a group of third-party agents that can perform the intended action;

generating, based on the group of third-party agents that can perform the intended action, a prompt that includes, for each of the third-party agents included in the group:

a corresponding selectable element for the third-party agent, and

a corresponding additional value that is specific to the third-party agent, the corresponding additional value being for an unspecified parameter that is unspecified by the user input;

transmitting, over one or more of the network interfaces, the prompt to the client device of the user, wherein transmitting the prompt to the client device of the user causes the prompt to be provided for presentation to the user via the automated assistant interface of the client device of the user;

receiving, over one or more of the network interfaces, and in response to the prompt, further user input transmitted by the client device of the user that indicates a selection of a particular third-party agent, of the third-party agents included in the group, through interaction with the corresponding selectable element for the particular third-party agent; and

in response to receiving the further user input that indicates the selection of the particular third-party agent:

facilitating, via one or more of the network interfaces, interactions between the client device and the particular third-party agent that occur via the automated assistant interface to perform the intended action.

2. The method of claim 1 , wherein the parameter for the intended action is a mandatory parameter.

3. The method of claim 2 , further comprising:

prior to generating the prompt that includes, for each of the third-party agents included in the group, the corresponding selectable element and the corresponding additional value:

generating, based on an additional value for an additional mandatory parameter for the intended action being unspecified, a prior prompt that solicits the additional value for the additional mandatory parameter;

transmitting, over one or more of the network interfaces, the prior prompt to the client device of the user, wherein transmitting the prior prompt to the client device of the user causes the prior prompt to be provided for presentation to the user via the automated assistant interface of the client device of the user; and

receiving, over one or more of the network interfaces, and in response to the prior prompt, prior user input transmitted by the client device of the user that indicates the additional value for the additional mandatory parameter.

4. The method of claim 3 , wherein the prior prompt that solicits the additional value for the additional mandatory parameter includes at least:

a prior selectable element associated with a first value, and

an additional prior selectable element associated with a second value,

wherein the prior user input indicates a corresponding selection of the first value or the second value, for use as the additional value for the additional mandatory parameter, through interaction with a respective one of the prior selectable element associated with the first value or the additional prior selectable element associated with the second value.

5. The method of claim 1 , wherein the parameter for the intended action is an optional parameter.

6. The method of claim 5 , further comprising:

prior to generating the prompt that includes, for each of the third-party agents included in the group, the corresponding selectable element and the corresponding additional value:

generating, based on an additional value for a mandatory parameter for the intended action being unspecified, a prior prompt that solicits the additional value for the mandatory parameter;

transmitting, over one or more of the network interfaces, the prior prompt to the client device of the user, wherein transmitting the prior prompt to the client device of the user causes the prior prompt to be provided for presentation to the user via the automated assistant interface of the client device of the user; and

receiving, over one or more of the network interfaces, and in response to the prior prompt, prior user input transmitted by the client device of the user that indicates the additional value for the mandatory parameter.

7. The method of claim 6 , wherein the prior prompt that solicits the additional value for the additional mandatory parameter includes at least:

a prior selectable element associated with a first value, and

an additional prior selectable element associated with a second value,

wherein the prior user input indicates a corresponding selection of the first value or the second value, for use as the additional value for the mandatory parameter, through interaction with a respective one of the prior selectable element associated with the first value or the additional prior selectable element associated with the second value.

8. The method of claim 1 , wherein the further user input is one of: spoken input, touch input, or typed input.

9. The method of claim 1 , wherein the corresponding additional value for the unspecified parameter that is specific to the third-party agent is based at least in part on the value for the parameter for the intended action.

10. The method of claim 1 , wherein facilitating the interactions between the client device and the particular third-party agent via the automated assistant interface to perform the intended action comprises:

receiving, over one or more of the network interfaces, and from the particular third-party agent, third-party content to be provided for presentation to the user via the automated assistant interface of the client device of the user; and

transmitting, over one or more of the network interfaces, the third-party content to the client device of the user, wherein transmitting the third-party content to the client device of the user causes the third-party content to be provided for presentation to the user via the automated assistant interface of the client device of the user.

11. The method of claim 10 , wherein facilitating the interactions between the client device and the particular third-party agent via the automated assistant interface to perform the intended action further comprises:

receiving, over one or more of the network interfaces, and in response to the third-party content being provided for presentation to the user, prior user input transmitted by the client device of the user that is received via the automated assistant interface of the client device of the user; and

transmitting, over one or more of the network interfaces, and to the particular the third-party, the prior user input.

12. A system comprising:

one or more processors; and

memory storing instructions that, when executed, cause one or more of the processors to:

receive, over one or more network interfaces, user input transmitted by a client device of a user, the user input being free-form and being directed to an automated assistant interface implemented by the client device;

determine, based on processing the user input, that the user input indicates an intended action and indicates a value for a parameter for the intended action;

select, based on the intended action and the value of the parameter, a group of third-party agents that can perform the intended action;

generate, based on the group of third-party agents that can perform the intended action, a prompt that includes, for each of the third-party agents included in the group:

a corresponding selectable element for the third-party agent, and

a corresponding additional value that is specific to the third-party agent, the corresponding additional value being for an unspecified parameter that is unspecified by the user input;

transmit, over one or more of the network interfaces, the prompt to the client device of the user, wherein transmitting the prompt to the client device of the user causes the prompt to be provided for presentation to the user via the automated assistant interface of the client device of the user;

receive, over one or more of the network interfaces, and in response to the prompt, further user input transmitted by the client device of the user that indicates a selection of a particular third-party agent, of the third-party agents included in the group, through interaction with the corresponding selectable element for the particular third-party agent; and

in response to receiving the further user input that indicates the selection of the particular third-party agent:

facilitate, via one or more of the network interfaces, interactions between the client device and the particular third-party agent that occur via the automated assistant interface to perform the intended.

13. The system of claim 12 , wherein the parameter for the intended action is a mandatory parameter.

14. The system of claim 13 , further comprising:

prior to generating the prompt that includes, for each of the third-party agents included in the group, the corresponding selectable element and the corresponding additional value:

generate, based on an additional value for an additional mandatory parameter for the intended action being unspecified, a prior prompt that solicits the additional value for the additional mandatory parameter;

transmit, over one or more of the network interfaces, the prior prompt to the client device of the user, wherein transmitting the prior prompt to the client device of the user causes the prior prompt to be provided for presentation to the user via the automated assistant interface of the client device of the user; and

receive, over one or more of the network interfaces, and in response to the prior prompt, prior user input transmitted by the client device of the user that indicates the additional value for the additional mandatory parameter.

15. The system of claim 14 , wherein the prior prompt that solicits the additional value for the additional mandatory parameter includes at least:

a prior selectable element associated with a first value, and

an additional prior selectable element associated with a second value,

wherein the prior user input indicates a corresponding selection of the first value or the second value, for use as the additional value for the additional mandatory parameter, through interaction with a respective one of the prior selectable element associated with the first value or the additional prior selectable element associated with the second value.

16. The system of claim 12 , wherein the further user input is one of: spoken input, touch input, or typed input.

17. The system of claim 12 , wherein the corresponding additional value for the unspecified parameter that is specific to the third-party agent is based at least in part on the value for the parameter for the intended action.

18. The system of claim 12 , wherein the instructions to facilitate the interactions between the client device and the particular third-party agent via the automated assistant interface to perform the intended action comprise instructions to:

receive, over one or more of the network interfaces, and from the particular third-party agent, third-party content to be provided for presentation to the user via the automated assistant interface of the client device of the user; and

transmit, over one or more of the network interfaces, the third-party content to the client device of the user, wherein transmitting the third-party content to the client device of the user causes the third-party content to be provided for presentation to the user via the automated assistant interface of the client device of the user.

19. The system of claim 18 , wherein the instructions to facilitate the interactions between the client device and the particular third-party agent via the automated assistant interface to perform the intended action further comprise instructions to:

receive, over one or more of the network interfaces, and in response to the third-party content being provided for presentation to the user, prior user input transmitted by the client device of the user that is received via the automated assistant interface of the client device of the user; and

transmit, over one or more of the network interfaces, and to the particular the third-party, the prior user input.

20. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to:

receive, over one or more network interfaces, user input transmitted by a client device of a user, the user input being free-form and being directed to an automated assistant interface implemented by the client device;

determine, based on processing the user input, that the user input indicates an intended action and indicates a value for a parameter for the intended action;

select, based on the intended action and the value of the parameter, a group of third-party agents that can perform the intended action;

generate, based on the group of third-party agents that can perform the intended action, a prompt that includes, for each of the third-party agents included in the group:

a corresponding selectable element for the third-party agent, and

a corresponding additional value that is specific to the third-party agent, the corresponding additional value being for an unspecified parameter that is unspecified by the user input;

transmit, over one or more of the network interfaces, the prompt to the client device of the user, wherein transmitting the prompt to the client device of the user causes the prompt to be provided for presentation to the user via the automated assistant interface of the client device of the user;

receive, over one or more of the network interfaces, and in response to the prompt, further user input transmitted by the client device of the user that indicates a selection of a particular third-party agent, of the third-party agents included in the group, through interaction with the corresponding selectable element for the particular third-party agent; and

in response to receiving the further user input that indicates the selection of the particular third-party agent:

facilitate, via one or more of the network interfaces, interactions between the client device and the particular third-party agent that occur via the automated assistant interface to perform the intended.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 26, 2021
From: KIRAZCI, ULAS; WANG, BO; CHEN, STEVE; VEMURI, SUNIL; JAMES, BARNABY; NYGAARD, VALERIE
To: GOOGLE INC.
Reel/Frame 055425/0879 →
CHANGE OF NAME Recorded Feb 26, 2021
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 055434/0988 →
Continuity (4)
Continuation 16877799 · May 19, 2020
Continuation 16244780 · Jan 10, 2019
Continuation 15394872 · Dec 30, 2016
Related Publication 20210134294A1 · May 6, 2021