IP Library › Granted Patent US 11,335,319
Granted Patent B2
US 11,335,319 · App. 16/894,604 · Granted May 17, 2022

Conversation-aware proactive notifications for a voice interface device

Inventors: Kenneth Mixter (Los Altos Hills, CA); Daniel Colish (Portland, OR); Tuan Nguyen (Mountain View, CA)
Assignee: Google LLC
G10L13/00G06F3/167G10L15/22G10L15/26H04L12/282H04L51/24H04L67/26G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,335,319
App. No.
16/894,604
Granted
May 17, 2022
Kind
B2
Abstract

A method for proactive notifications in a voice interface device includes: receiving a first user voice request for an action with an future performance time; assigning the first user voice request to a voice assistant service for performance; subsequent to the receiving, receiving a second user voice request and in response to the second user voice request initiating a conversation with the user; and during the conversation: receiving a notification from the voice assistant service of performance of the action; triggering a first audible announcement to the user to indicate a transition from the conversation and interrupting the conversation; triggering a second audible announcement to the user to indicate performance of the action; and triggering a third audible announcement to the user to indicate a transition back to the conversation and rejoining the conversation.

Claims (97)

1. A method implemented at an electronic voice interface device including a speaker, one or more processors, and memory storing instructions for execution by the one or more processors, the method comprising:

receiving a first voice request;

assigning the first voice request to a server-implemented voice assistant service;

subsequent to the receiving of the first voice request:

receiving a second voice request; and

assigning the second voice request to the server-implemented voice assistant service as part of an interaction; and

subsequent to the receiving of the second voice request:

receiving from the server-implemented voice assistant service a response to the first voice request;

receiving from the server-implemented voice assistant service a response to the second voice request as part of the interaction;

triggering an audible announcement indicating a transition from the interaction; and

following the audible announcement and prior to triggering any announcements related to the interaction including the response to the second voice request, triggering a verbal announcement including the response to the first voice request.

2. The method of claim 1 , further comprising:

following the triggering of the verbal announcement including the response to the first voice request, triggering an audible announcement indicating a transition back to the user interaction; and

following the triggering of the audible announcement indicating a transition back to the user interaction, triggering a verbal announcement related to the user interaction including the response to the second voice request.

3. The method of claim 1 , wherein:

the first voice request is a request for an action with a future performance time;

the response to the first voice request includes a notification indicating performance of the action;

the second voice request is a request for information;

the response to the second voice request includes the requested information; and

the electronic voice interface device receives the second voice request prior to the future performance time.

4. The method of claim 1 , further comprising determining based on context of the user interaction an appropriate time at which to trigger the audible announcement indicating a transition from the user interaction.

5. The method of claim 1 , wherein:

the user interaction is performed between a user of the electronic voice interface device and a software agent; and

the software agent determines and generates the audible announcement indicating a transition from the user interaction.

6. The method of claim 5 , wherein:

the software agent performs operations related to satisfaction of the second voice request; and

an agent module manages interactions between the software agent and the user during the user interaction.

7. The method of claim 6 , wherein:

the agent module comprises a library of transition-in phrases and a library of transition-out phrases;

the agent module generates content of the audible announcement indicating a transition from the user interaction from one or more of the transition-in phrases; and

the agent module generates content of the audible announcement indicating a transition back to the user interaction from one or more of the transition-out phrases.

8. The method of claim 7 , wherein a conversation manager module causes the agent module to:

interrupt the user interaction;

trigger the audible announcement indicating a transition from the user interaction;

wait for the conversation manager module to trigger the verbal announcement including the response to the first voice request;

trigger the audible announcement indicating a transition back to the user interaction upon completion of the verbal announcement including the response to the first voice request; and

rejoin the user interaction.

9. An electronic voice interface device comprising a speaker, one or more processors, and memory storing executable instructions that are configured to cause the one or more processors to perform operations including:

receiving a first voice request;

assigning the first voice request to a server-implemented voice assistant service;

subsequent to the receiving of the first voice request:

receiving a second voice request; and

assigning the second voice request to the server-implemented voice assistant service as part of a user interaction; and

subsequent to the receiving of the second voice request:

receiving from the server-implemented voice assistant service a response to the first voice request;

receiving from the server-implemented voice assistant service a response to the second voice request as part of the user interaction;

triggering an audible announcement indicating a transition from the user interaction; and

following the audible announcement and prior to triggering any announcements related to the user interaction including the response to the second voice request, triggering a verbal announcement including the response to the first voice request.

10. The electronic voice interface device of claim 9 , wherein the instructions are further configured to cause the one or more processors to perform:

following the triggering of the verbal announcement including the response to the first voice request, triggering an audible announcement indicating a transition back to the user interaction; and

following the triggering of the audible announcement indicating a transition back to the user interaction, triggering a verbal announcement related to the user interaction including the response to the second voice request.

11. The electronic voice interface device of claim 9 , wherein:

the first voice request is a request for an action with a future performance time;

the response to the first voice request includes a notification indicating performance of the action;

the second voice request is a request for information;

the response to the second voice request includes the requested information; and

the electronic voice interface device receives the second voice request prior to the future performance time.

12. The electronic voice interface device of claim 9 , wherein the instructions are further configured to cause the one or more processors to determine based on context of the user interaction an appropriate time at which to trigger the audible announcement indicating a transition from the user interaction.

13. The electronic voice interface device of claim 9 , wherein:

the user interaction is performed between a user of the electronic voice interface device and a software agent; and

the software agent determines and generates the audible announcement indicating a transition from the user interaction.

14. The electronic voice interface device of claim 13 , wherein:

the software agent performs operations related to satisfaction of the second voice request;

an agent module manages interactions between the software agent and the user during the user interaction;

the agent module comprises a library of transition-in phrases and a library of transition-out phrases;

the agent module generates content of the audible announcement indicating a transition from the user interaction from one or more of the transition-in phrases; and

the agent module generates content of the audible announcement indicating a transition back to the user interaction from one or more of the transition-out phrases.

15. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic voice interface device with one or more processors, cause the one or more processors to perform:

receiving a first voice request;

assigning the first voice request to a server-implemented voice assistant service;

subsequent to the receiving of the first voice request:

receiving a second voice request; and

assigning the second voice request to the server-implemented voice assistant service as part of a user interaction; and

subsequent to the receiving of the second voice request:

receiving from the server-implemented voice assistant service a response to the first voice request;

receiving from the server-implemented voice assistant service a response to the second voice request as part of the user interaction;

triggering an audible announcement indicating a transition from the user interaction; and

following the audible announcement and prior to triggering any announcements related to the user interaction including the response to the second voice request, triggering a verbal announcement including the response to the first voice request.

16. The non-transitory computer readable storage medium of claim 15 , wherein the instructions are further configured to cause the one or more processors to perform:

following the triggering of the verbal announcement including the response to the first voice request, triggering an audible announcement indicating a transition back to the user interaction; and

following the triggering of the audible announcement indicating a transition back to the user interaction, triggering a verbal announcement related to the user interaction including the response to the second voice request.

17. The non-transitory computer readable storage medium of claim 15 , wherein:

the first voice request is a request for an action with a future performance time;

the response to the first voice request includes a notification indicating performance of the action;

the second voice request is a request for information;

the response to the second voice request includes the requested information; and

the electronic voice interface device receives the second voice request prior to the future performance time.

18. The non-transitory computer readable storage medium of claim 15 , wherein the instructions are further configured to cause the one or more processors to determine based on context of the user interaction an appropriate time at which to trigger the audible announcement indicating a transition from the user interaction.

19. The non-transitory computer readable storage medium of claim 15 , wherein:

the user interaction is performed between a user of the electronic voice interface device and a software agent; and

the software agent determines and generates the audible announcement indicating a transition from the user interaction.

20. The non-transitory computer readable storage medium of claim 19 , wherein:

the software agent performs operations related to satisfaction of the second voice request;

an agent module manages interactions between the software agent and the user during the user interactions;

the agent module comprises a library of transition-in phrases and a library of transition-out phrases;

the agent module generates content of the audible announcement indicating a transition from the user interaction from one or more of the transition-in phrases; and

the agent module generates content of the audible announcement indicating a transition back to the user interaction from one or more of the transition-out phrases.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2021
From: MIXTER, KENNETH; COLISH, DANIEL
To: GOOGLE LLC
Reel/Frame 057637/0047 →
Continuity (3)
Continuation 15841284 · Dec 13, 2017
Provisional Application 62441116 · Dec 30, 2016
Related Publication 20200302912A1 · Sep 24, 2020