Granted Patent B2

US 12,348,915 · App. 18/127,465 · Granted Jul 1, 2025

Inventors: Andrea Terwisscha van Scheltinga (Zurich, CH); Zaheed Sabur (Baar, CH); Michael Reutov (Zug, CH); Pratik Gilda (Zurich, CH)

Assignee: GOOGLE LLC

H04Q3/0033G06F3/167G10L15/22H04W8/205H04W36/0094H04W76/15H04L67/535H04Q2213/13547H04W36/247

Abstract

Claims (57)

1. A method implemented using one or more processors of a first computing device, the method comprising:

receiving audio data that captures spoken user input of a user of the first computing device, the audio data being generated via one or more microphones of the first computing device, the spoken user input being directed to an assistant executing at least in part at the first computing device;

processing the spoken user input locally at the first computing device to identify a request that is included in the spoken user input;

determining a response that is responsive to the request included in the spoken user input;

inferring, based on the request that is included in the spoken user input, an activity being performed by the user;

determining, based on the request that is included in the spoken user input, whether to provide audible output, that captures the response that is responsive to the request included in the spoken user input, at the first computing device or an additional computing device that is in communication with the first computing device,

wherein determining whether to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the first computing device or the additional computing device that is in communication with the first computing device is based on the activity being performed by the user; and

in response to determining to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the additional computing device that is in communication with the first computing device:

causing the additional computing device to render the audible output in response to receiving the spoken user input.

2. The method of claim 1 , wherein the request that is included in the spoken user input requests directions to a location, wherein the response that is responsive to the request included in the spoken user input includes the directions to the location, and wherein the additional computing device is a vehicle computing device.

3. The method of claim 2 , wherein determining to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the additional computing device that is in communication with the first computing device is further based on the request that is included in the spoken user input requesting the directions to the location.

4. The method of claim 2 , wherein determining whether to provide audible output, that captures the response that is responsive to the request included in the spoken user input, at the first computing device or the additional computing device that is in communication with the first computing device is further based on detecting a connection between the first computing device and the vehicle computing device.

5. The method of claim 2 , wherein determining the response that is responsive to the request included in the spoken user input comprises:

transmitting, to a server, a query that requests the directions to the location; and

receiving, from the server, the directions to the location.

6. The method of claim 2 , wherein determining the response that is responsive to the request included in the spoken user input comprises:

transmitting, to a server, a query that requests the directions to the location; and

causing the server to transmit the directions to the location directly to the vehicle computing device.

7. The method of claim 1 , further comprising:

identifying a plurality of additional computing devices that are in communication with the first computing device, the plurality of additional computing devices including at least the additional computing device and a further additional computing device; and

selecting, based on the activity being performed by the user, the additional computing device to provide the audible output that captures the response that is responsive to the request included in the spoken user input, and in lieu of the first computing device or the further additional computing device.

8. The method of claim 7 , wherein the first computing device is a standalone speaker associated with the user, wherein the additional computing device is a phone associated with the user, and wherein the further additional computing device is a TV associated with the user.

9. The method of claim 1 , wherein the request does not explicitly identify the additional computing device.

10. A first computing device, the first computing device comprising:

one or more first computing device microphones;

at least one first computing device processor; and

first computing device memory storing first computing device instructions that, when executed, cause the at least one first computing device processor to:

receive audio data that captures spoken user input of a user of the first computing device, the audio data being generated via one or more microphones of the first computing device, the spoken user input being directed to an assistant executing at least in part at the first computing device;

process the spoken user input locally at the first computing device to identify a request that is included in the spoken user input;

determine a response that is responsive to the request included in the spoken user input;

infer, based on the request that is included in the spoken user input, an activity being performed by the user;

determine, based on the request that is included in the spoken user input, whether to provide audible output, that captures the response that is responsive to the request included in the spoken user input, at the first computing device or an additional computing device that is in communication with the first computing device,

wherein the first computing device instructions to determine whether to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the first computing device or the additional computing device that is in communication with the first computing device are executed based on the activity being performed by the user; and

in response to determining to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the additional computing device that is in communication with the first computing device:

cause the additional computing device to render the audible output in response to receiving the spoken user input.

11. The first computing device of claim 10 , wherein the request that is included in the spoken user input requests directions to a location, wherein the response that is responsive to the request included in the spoken user input includes the directions to the location, and wherein the additional computing device is a vehicle computing device.

12. The first computing device of claim 11 , wherein the first computing device instructions to determine to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the additional computing device that is in communication with the first computing device are further executed based on the request that is included in the spoken user input requesting the directions to the location.

13. The first computing device of claim 11 , wherein the first computing device instructions to determine whether to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the first computing device or the additional computing device that is in communication with the first computing device are further executed based on detecting a connection between the first computing device and the vehicle computing device.

14. The first computing device of claim 11 , wherein the first computing device instructions to determine the response that is responsive to the request included in the spoken user input comprise first computing device instructions to:

transmit, to a server, a query that requests the directions to the location; and

receive, from the server, the directions to the location.

15. The first computing device of claim 11 , wherein the first computing device instructions to determine the response that is responsive to the request included in the spoken user input comprise first computing device instructions to:

transmit, to a server, a query that requests the directions to the location; and

cause the server to transmit the directions to the location directly to the vehicle computing device.

16. The first computing device of claim 10 , wherein the first computing device instructions further comprise first computing device instructions to:

identify a plurality of additional computing devices that are in communication with the first computing device, the plurality of additional computing devices including at least the additional computing device and a further additional computing device; and

select, based on the activity being performed by the user, the additional computing device to provide the audible output that captures the response that is responsive to the request included in the spoken user input, and in lieu of the first computing device or the further additional computing device.

17. The first computing device of claim 16 , wherein the first computing device is a standalone speaker associated with the user, wherein the additional computing device is a phone associated with the user, and wherein the further additional computing device is a TV associated with the user.

18. A non-transitory computer-readable storage medium of a first computing device storing first computing device instructions that, when executed by one or more first computing device hardware processors, cause the one or more first computing device hardware processors to perform first computing device operations, the first computing device operations comprising:

receiving audio data that captures spoken user input of a user of the first computing device, the audio data being generated via one or more microphones of the first computing device, the spoken user input being directed to an assistant executing at least in part at the first computing device;

processing the spoken user input locally at the first computing device to identify a request that is included in the spoken user input;

determining a response that is responsive to the request included in the spoken user input;

inferring, based on the request that is included in the spoken user input, an activity being performed by the user;

determining, based on the request that is included in the spoken user input, whether to provide audible output, that captures the response that is responsive to the request included in the spoken user input, at the first computing device or an additional computing device that is in communication with the first computing device,

wherein determining whether to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the first computing device or the additional computing device that is in communication with the first computing device is based on the activity being performed by the user; and

in response to determining to provide the audible output, that captures the response that is responsive to the request included in the spoken user input, at the additional computing device that is in communication with the first computing device:

causing the additional computing device to render the audible output in response to receiving the spoken user input.

Assignments (1)

Continuity (7)

References Cited (62)