Granted Patent B2

US 12,254,038 · App. 18/538,773 · Granted Mar 18, 2025

Inventors: Matthew Sharifi (Kilchberg, CH); Victor Carbune (Zurich, CH)

Assignee: GOOGLE LLC

G06F16/433G06F3/167G06F40/30G10L15/22G10L2015/223

Abstract

Claims (76)

1. A method implemented by one or more processors, the method comprising:

receiving, via a client device of a user, user input directed to an automated assistant, the user input including an assistant command;

determining, based on the assistant command, whether first-party server data or third-party server data is needed to perform a certain fulfillment of the assistant command;

in response to determining that the first-party server data is needed to perform the certain fulfillment of the assistant command:

transmitting a first-party request to a first-party server to cause the certain fulfillment of the assistant command to be performed; and

in response to determining that the third-party server data is needed to perform the certain fulfillment of the assistant command:

generating a prompt that requests the user consent to transmitting a third-party request to a third-party server, wherein the request to be transmitted to the third-party server requests the third-party server data that is needed to perform the certain fulfillment of the assistant command;

causing the prompt to be provided for presentation to the user via the client device;

receiving, via the client device of the user, additional user input directed to the automated assistant, the additional user input being responsive to the prompt;

determining, based on the additional user input, whether to transmit the third-party request to the third-party server to cause the certain fulfillment of the assistant command to be performed.

2. The method of claim 1 , further comprising:

in response to determining the additional user input does not consent to transmitting the third-party request to the third-party server to cause the certain fulfillment of the assistant command to be performed:

refraining from causing the third-party request to be transmitted to the third-party server to cause the certain fulfillment of the assistant command to be performed; and

causing the client device to generate client data in an attempt to perform the certain fulfillment of the assistant command.

3. The method of claim 2 , further comprising:

determining whether the certain fulfillment of the assistant command can be performed utilizing the client data; and

in response to determining that the certain fulfillment of the assistant command cannot be performed utilizing the client data:

generating a notification that indicates the certain fulfillment of the assistant command cannot be performed without the third-party server data; and

causing the notification to be provided for presentation to the user via the client device.

4. The method of claim 3 , further comprising:

utilizing the client data to perform an alternate fulfillment of the assistant command, wherein the alternate fulfillment of the assistant command is performed in the attempt to perform the certain fulfillment of the assistant command.

5. The method of claim 3 , further comprising:

in response to determining that the certain fulfillment of the assistant command can be performed utilizing the client data:

utilizing the client data to perform the certain fulfillment of the assistant command.

6. The method of claim 1 , further comprising:

in response to determining the additional user input consents to transmitting the third-party request to the third-party server to cause the certain fulfillment of the assistant command to be performed:

causing the third-party request to be transmitted to the third-party server to obtain the third-party server data to cause the certain fulfillment of the assistant command to be performed.

7. The method of claim 1 , wherein determining that the first-party server data is needed to perform the certain fulfillment of the assistant command comprises determining that the assistant command is a first-party assistant command that is associated with a first-party.

8. The method of claim 1 , wherein determining that the third-party server data is needed to perform the certain fulfillment of the assistant command comprises determining that the assistant command is a third-party assistant command that is associated with a third-party.

9. The method of claim 1 , wherein causing the prompt to be provided for presentation to the user via the client device comprises:

causing synthesized speech audio data that includes the prompt to be audibly rendered for presentation to the user via one or more speakers of the client device; and/or

causing the prompt and one or more corresponding selectable elements to be visually rendered for presentation to the user via a display of the client device.

10. A system comprising:

at least one processor; and

memory storing instructions that, when executed, cause the at least one processor to be operable to:

receive, via a client device of a user, user input directed to an automated assistant, the user input including an assistant command;

determine, based on the assistant command, whether first-party server data or third-party server data is needed to perform a certain fulfillment of the assistant command;

in response to determining that the first-party server data is needed to perform the certain fulfillment of the assistant command:

transmit a first-party request to a first-party server to cause the certain fulfillment of the assistant command to be performed; and

in response to determining that the third-party server data is needed to perform the certain fulfillment of the assistant command:

generate a prompt that requests the user consent to transmitting a third-party request to a third-party server, wherein the request to be transmitted to the third-party server requests the third-party server data that is needed to perform the certain fulfillment of the assistant command;

cause the prompt to be provided for presentation to the user via the client device;

receive, via the client device of the user, additional user input directed to the automated assistant, the additional user input being responsive to the prompt;

determine, based on the additional user input, whether to transmit the third-party request to the third-party server to cause the certain fulfillment of the assistant command to be performed.

11. The system of claim 10 , wherein the at least one processor is further operable to:

in response to determining the additional user input does not consent to transmitting the third-party request to the third-party server to cause the certain fulfillment of the assistant command to be performed:

refrain from causing the third-party request to be transmitted to the third-party server to cause the certain fulfillment of the assistant command to be performed; and

cause the client device to generate client data in an attempt to perform the certain fulfillment of the assistant command.

12. The system of claim 11 , wherein the at least one processor is further operable to:

determine whether the certain fulfillment of the assistant command can be performed utilizing the client data; and

in response to determining that the certain fulfillment of the assistant command cannot be performed utilizing the client data:

generate a notification that indicates the certain fulfillment of the assistant command cannot be performed without the third-party server data; and

cause the notification to be provided for presentation to the user via the client device.

13. The system of claim 12 , wherein the at least one processor is further operable to:

utilizing the client data to perform an alternate fulfillment of the assistant command, wherein the alternate fulfillment of the assistant command is performed in the attempt to perform the certain fulfillment of the assistant command.

14. The system of claim 12 , wherein the at least one processor is further operable to:

in response to determining that the certain fulfillment of the assistant command can be performed utilizing the client data:

utilize the client data to perform the certain fulfillment of the assistant command.

15. The system of claim 10 , wherein the at least one processor is further operable to:

in response to determining the additional user input consents to transmitting the third-party request to the third-party server to cause the certain fulfillment of the assistant command to be performed:

cause the third-party request to be transmitted to the third-party server to obtain the third-party server data to cause the certain fulfillment of the assistant command to be performed.

16. The system of claim 10 , wherein, in determining that the first-party server data is needed to perform the certain fulfillment of the assistant command, the at least one processor is operable to determine that the assistant command is a first-party assistant command that is associated with a first-party.

17. The system of claim 10 , wherein, in determining that the third-party server data is needed to perform the certain fulfillment of the assistant command, the at least one processor is operable to determine that the assistant command is a third-party assistant command that is associated with a third-party.

18. The system of claim 10 , wherein, in causing the prompt to be provided for presentation to the user via the client device, the at least one processor is operable to:

cause synthesized speech audio data that includes the prompt to be audibly rendered for presentation to the user via one or more speakers of the client device; and/or

cause the prompt and one or more corresponding selectable elements to be visually rendered for presentation to the user via a display of the client device.

19. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to be operable to perform operations, the operations comprising:

receiving, via a client device of a user, user input directed to an automated assistant, the user input including an assistant command;

determining, based on the assistant command, whether first-party server data or third-party server data is needed to perform a certain fulfillment of the assistant command;

in response to determining that the first-party server data is needed to perform the certain fulfillment of the assistant command:

transmitting a first-party request to a first-party server to cause the certain fulfillment of the assistant command to be performed; and

in response to determining that the third-party server data is needed to perform the certain fulfillment of the assistant command:

generating a prompt that requests the user consent to transmitting a third-party request to a third-party server, wherein the request to be transmitted to the third-party server requests the third-party server data that is needed to perform the certain fulfillment of the assistant command;

causing the prompt to be provided for presentation to the user via the client device;

receiving, via the client device of the user, additional user input directed to the automated assistant, the additional user input being responsive to the prompt;

determining, based on the additional user input, whether to transmit the third-party request to the third-party server to cause the certain fulfillment of the assistant command to be performed.

Assignments (1)

Continuity (2)

References Cited (9)