Granted Patent B1

US 10,009,666 · App. 15/802,568 · Granted Jun 26, 2018

Inventors: Andrea Terwisscha van Scheltinga (Zurich, CH); Zaheed Sabur (Baar, CH); Michael Reutov (Zug, CH); Pratik Gilda (Zürich, CH)

Assignee: GOOGLE LLC

H04Q3/0033H04W36/0094H04W76/025H04W76/15H04Q2213/13547

Abstract

Claims (93)

1. A method comprising:

receiving, by a computing system, an indication of a query received by a first computing device from a user, wherein the query forms at least a part of a conversation between the user and a first assistant executing at the first computing device;

determining, by the computing system, whether to handoff the conversation from the first assistant executing at the first computing device to a second assistant executing at a second computing device, wherein determining whether to handoff the conversation is based at least in part on a difference in functional capabilities between the first computing device and the second computing device, and wherein determining whether to handoff the conversation is based at least in part on a difference in functional capabilities between the first computing device and the second computing device comprises:

determining, by the computing system, that a response to the query is to be visually outputted, and

in response to determining that the first computing device is not operably coupled to a first display device, and that the second computing device is operably coupled to a second display device, determining, by the computing system, to handoff the conversation to the second assistant executing at the second computing device; and

in response to determining to handoff the conversation to the second assistant executing at the second computing device, sending, by the computing system to the second computing device, an indication of the conversation.

2. The method of claim 1 , further comprising:

selecting, by the computing system, the second computing device to receive the handoff of the conversation from a plurality of computing devices associated with the user that are available to receive the handoff of the conversation.

3. The method of claim 2 , wherein selecting the second computing device to receive the handoff is based at least in part on a physical proximity of the second computing device to the first computing device.

4. The method of claim 1 , further comprising:

determining, by the computing system, contextual information associated with the conversation between the user and the first assistant executing at the first computing device;

receiving, by the computing system, an indication of a second query received by the second computing device from the user, wherein the query comprises a first query;

determining, by the computing system and based at least in part on the contextual information, that the second query refers back to previous portions of the conversation between the user and the first assistant executing at the first computing device that took place prior to handoff of the conversation to the second assistant executing at the second computing device; and

determining, by the computing system, a response to the second query based at least in part on the contextual information associated with the conversation between the user and the first assistant executing at the first computing device.

5. The method of claim 1 , further comprising:

receiving, by the computing system from a third party application, an indication of whether to handoff the conversation from the first assistant executing at the first computing device to the second assistant executing at a second computing device; and

determining, by the computing system, whether to handoff the conversation from the first assistant executing at the first computing device to the second assistant executing at a second computing device based on the indication of whether to handoff the conversation from the first assistant executing at the first computing device to the second assistant executing at a second computing device.

6. The method of claim 1 , wherein the query includes an explicit command to handoff the conversation.

7. The method of claim 1 , wherein the computing system receives the indication of the query from the first computing device.

8. The method of claim 7 , wherein the first computing device receives user input indicative of the query.

9. A system comprising:

a first computing device;

a second computing device; and

a digital assistant system operably connected to the first computing device and the second computing device via a network;

wherein the first computing device is operable to:

receive user input indicative of a query that forms at least a part of a conversation between a user and a first assistant executing at the first computing device, and

send an indication of the query to the digital assistant system;

wherein the digital assistant system is operable to:

receive the indication of the query from the first computing device,

determine whether to handoff the conversation from the first assistant executing at the first computing device to a second assistant executing at the second computing device, wherein the digital assistant system is further operable to:

determine whether to handoff the conversation based at least in part on a difference in functional capabilities between the first computing device and the second computing device,

determine that the response to the query is to be visually outputted, and

in response to determining that the first computing device is not operably coupled to a first display device, and that the second computing device is operably coupled to a second display device, determine to handoff the conversation to the second assistant executing at the second computing device, and

in response to determining to handoff the conversation to the second assistant executing at the second computing device, send to the second computing device an indication of the conversation; and

wherein the second computing device is operable to:

receive, from the digital assistant system, the indication of the conversation, and

output a response to the query as part of the conversation.

10. The system of claim 9 , wherein the digital assistant system is further operable to: select the second computing device to receive the handoff of the conversation from a plurality of computing devices associated with the user that are available to receive the handoff of the conversation.

11. The system of claim 9 , wherein:

the digital assistant system is further operable to determine contextual information associated with the conversation between the user and the first assistant executing at the first computing device;

the second computing device is further operable to:

receive a second user input indicative of a second query from the user, wherein the user input comprises a first user input, and wherein the query comprises a first query, and

send an indication of the second query to the digital assistant system;

the digital assistant system is further operable to:

determine, based at least in part on the contextual information, that the second query refers back to previous portions of the conversation between the user and the first assistant executing at the first computing device that took place prior to handoff of the conversation to the second assistant executing at the second computing device,

determine a second response to the second query based at least in part on the contextual information associated with the conversation between the user and the first assistant executing at the first computing device, wherein the response comprises a first response, and

send an indication of the second response to the second query to the second computing device; and

the second computing device is further operable to:

receive the indication of the second response to the second query from the digital assistant system, and

output the second response to the second query.

12. The system of claim 9 , wherein the digital assistant system is further configured to:

receive, from a third party application, an indication of whether to handoff the conversation from the first assistant executing at the first computing device to the second assistant executing at a second computing device; and

determine whether to handoff the conversation from the first assistant executing at the first computing device to the second assistant executing at a second computing device based on the indication of whether to handoff the conversation from the first assistant executing at the first computing device to the second assistant executing at a second computing device.

13. The system of claim 9 , wherein the query includes an explicit command to handoff the conversation.

14. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause at least one processor of a computing system to:

receive an indication of a query received by a first computing device from a user, wherein the query forms at least a part of a conversation between the user and a first assistant executing at the first computing device;

determine whether to handoff the conversation from the first assistant executing at the first computing device to a second assistant executing at a second computing device based at least in part on a difference in functional capabilities between the first computing device and the second computing device, wherein the instructions that cause the at least one processor to determine whether to handoff the conversation is based at least in part on the difference in functional capabilities between the first computing device and the second computing device comprise instructions to:

determine that a response to the query is to be visually outputted, and

in response to determining that the first computing device is not operably coupled to a first display device, and that the second computing device is operably coupled to a second display device, determine to handoff the conversation to the second assistant executing at the second computing device; and

in response to determining to handoff the conversation to the second assistant executing at the second computing device, send an indication of the conversation to the second computing device.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the instructions, when executed, further cause the at least one processor of the computing system to:

determine a form in which to output the response; and

select the second computing device to receive the handoff of the conversation from a plurality of computing devices associated with the user that are available to receive the handoff of the conversation and that are capable of outputting the response in the determined form.

16. A method comprising:

receiving, by a computing system, an indication of a query received by a first computing device from a user, wherein the query forms at least a part of a conversation between the user and a first assistant executing at the first computing device;

determining, by the computing system, contextual information associated with the conversation between the user and the first assistant executing at the first computing device;

determining, by the computing system, whether to handoff the conversation from the first assistant executing at the first computing device to a second assistant executing at a second computing device;

in response to determining to handoff the conversation to the second assistant executing at the second computing device, sending, by the computing system to the second computing device, an indication of the conversation;

receiving, by the computing system, an indication of a second query received by the second computing device from the user, wherein the query comprises a first query;

determining, by the computing system and based at least in part on the contextual information, that the second query refers back to previous portions of the conversation between the user and the first assistant executing at the first computing device that took place prior to handoff of the conversation to the second assistant executing at the second computing device; and

determining, by the computing system, a response to the second query based at least in part on the contextual information associated with the conversation between the user and the first assistant executing at the first computing device.

17. A system comprising:

a first computing device;

a second computing device; and

a digital assistant system operably connected to the first computing device and the second computing device via a network;

wherein the first computing device is operable to:

receive a first user input indicative of a first query that forms at least a part of a conversation between a user and a first assistant executing at the first computing device, and

send an indication of the first query to the digital assistant system; wherein the digital assistant system is operable to:

receive the indication of the first query from the first computing device,

determine contextual information associated with the conversation between the user and the first assistant executing at the first computing device,

determine whether to handoff the conversation from the first assistant executing at the first computing device to a second assistant executing at the second computing device, and

in response to determining to handoff the conversation to the second assistant executing at the second computing device, send to the second computing device an indication of the conversation;

wherein the second computing device is operable to:

receive, from the digital assistant system, the indication of the conversation,

output a response to the first query as part of the conversation,

receive a second user input indicative of a second query from the user that forms at least a part of the conversation between the user and the second assistant executing at the second computing device, and

send an indication of the second query to the digital assistant system; wherein the digital assistant system is further operable to:

determine, based at least in part on the contextual information, that the second query refers back to previous portions of the conversation between the user and the first assistant executing at the first computing device that took place prior to handoff of the conversation to the second assistant executing at the second computing device,

determine a second response to the second query based at least in part on the contextual information associated with the conversation between the user and the first assistant executing at the first computing device, wherein the response comprises a first response, and

send an indication of the second response to the second query to the second computing device; and

wherein the second computing device is further operable to:

receive the indication of the second response to the second query from the digital assistant system, and

output the second response to the second query.

Assignments (1)

Continuity (2)

Cited By (10)

US 12,288,563 US 12,333,404 US 12,348,915 US 12,386,434 US 12,388,926 US 12,456,470 US 12,469,033 US 12,477,470 US 12,608,171 US 12,748,568