Granted Patent B1

US 12,327,550 · App. 17/847,534 · Granted Jun 10, 2025

Inventors: Zeya Chen (Charlottesville, VA); Charles Edwin Ashton Brett (Seattle, WA); Jay Patel (Seattle, WA); Lizhen Peng (Seattle, WA); Aniruddha Basak (Seattle, WA); Hongyang Wang (Seattle, WA); Yunfeng Jiang (Seattle, WA); Sven Eberhardt (Seattle, WA); Akshay Kumar (Seattle, WA); William Evan Welbourne (Seattle, WA); Sara Hillenmeyer (Seattle, WA)

Assignee: Amazon Technologies, Inc.

G10L15/1815G06F3/167G06F40/30G06N7/01G10L15/22H04L41/0893G10L2015/223H04L67/306

Abstract

Claims (72)

1. A method, comprising:

generating first data indicating a directive sent to at least one of a voice interface device or an accessory device associated with user account data, the directive causing the accessory device to perform an action responsive to a user utterance;

storing use data indicating past instances of the directive being sent to the at least one of the voice interface device or the accessory device;

storing user identifier information representing attributes of user utterances associated with a user of the user account data;

generating second data representing a recommendation specific to the user requesting that the user speak the user utterance based at least in part on the use data and the first data, wherein the recommendation is configured to be output by the voice interface device to request that the user speak the user utterance as the user has provided prior to the recommendation, the recommendation further indicating that the user has previously provided the user utterance prior to the recommendation;

determining, from user input data and detecting the attributes of the user utterances associated with the user, that the user is interacting with the voice interface device; and

causing the recommendation to be output by the voice interface device based at least in part on determining that the user is interacting with the voice interface device.

2. The method of claim 1 , further comprising:

determining a directive type associated with the directive;

determining that the directive type is predefined as being a permitted type for sending recommendations; and

wherein generating the second data representing the recommendation comprises generating the second data representing the recommendation based at least in part on the directive type being the permitted type.

3. The method of claim 1 , further comprising:

determining a time at which the user is interacting with the voice interface device;

determining that the time is within a time range predefined as being associated with sending recommendations; and

wherein causing the recommendation to be output comprises causing the recommendation to be output based at least in part on the time being within the time range.

4. The method of claim 1 , further comprising sending the second data to a user device associated with the user, the second data causing display of the recommendation on the user device, wherein the recommendation, when displayed on the user device, includes an indication of why the recommendation is being surfaced on the user device.

5. The method of claim 1 , further comprising sending the second data to a user device associated with the user, the second data causing display of the recommendation on the user device, wherein the recommendation, when displayed on the user device, includes an option that, when selected, causes the action to be performed.

6. The method of claim 1 , further comprising:

sending the second data to a user device associated with the user, the second data causing display of the recommendation on the user device, wherein the recommendation, when displayed on the user device, includes an option that, when selected, indicates subsequent recommendations are not to be sent;

receiving user input data indicating selection of the option; and

including an indication of the user input data in the use data such that the subsequent recommendations are not sent to the user device.

7. The method of claim 1 , wherein:

the use data indicates that the past instances occurred during a period of time;

the first data indicates that the directive was sent during the period of time; and

determining that the user is interacting with the voice interface device occurs during the period of time.

8. The method of claim 1 , further comprising:

determining that the user account data indicates occasional repeat actions are enabled; and

wherein causing the recommendation to be output comprises causing the recommendation to be output based at least in part on the user account data indicating the occasional repeat actions are enabled.

9. The method of claim 1 , further comprising:

receiving audio data from the voice interface device;

determining, from the audio data, that the user is in proximity to the voice interface device; and

wherein determining that the user is interacting with the voice interface device comprises determining that the user is interacting with the voice interface device based at least in part on the user being in proximity to the voice interface device.

10. The method of claim 1 , further comprising:

determining that the voice interface device is utilized to control operation of the accessory device more than other voice interface devices associated with the user account data; and

selecting, based at least in part on the voice interface device being utilized to control operation of the accessory device more than the other voice interface devices, the accessory device to perform the action in response to receiving user input data accepting the recommendation.

11. A system comprising:

one or more processors; and

non-transitory computer-readable media storing instructions that, when executed by the one or more processors, causes the one or more processors to perform operations comprising:

generating first data indicating a directive sent to at least one of a voice interface device or an accessory device associated with user account data, the directive causing the accessory device to perform an action responsive to a user utterance;

storing use data indicating past instances of the directive being sent to the at least one of the voice interface device or the accessory device;

storing user identifier information representing attributes of user utterances associated with a user of the user account data;

generating second data representing a recommendation specific to the user requesting that the user speak the user utterance based at least in part on the use data and the first data, wherein the recommendation is configured to be output by the voice interface device to request that the user speak the user utterance as the user has provided prior to the recommendation, the recommendation further indicating that the user has previously provided the user utterance prior to the recommendation;

determining, from user input data and detecting the attributes of the user utterances associated with the user, that the user is interacting with the voice interface device; and

causing the recommendation to be output by the voice interface device based at least in part on determining that the user is interacting with the voice interface device.

12. The system of claim 11 , the operations further comprising:

determining a directive type associated with the directive;

determining that the directive type is predefined as being a permitted type for sending recommendations; and

wherein generating the second data representing the recommendation comprises generating the second data representing the recommendation based at least in part on the directive type being the permitted type.

13. The system of claim 11 , the operations further comprising:

determining a time at which the user is interacting with the voice interface device;

determining that the time is within a time range predefined as being associated with sending recommendations; and

wherein causing the recommendation to be output comprises causing the recommendation to be output based at least in part on the time being within the time range.

14. The system of claim 11 , the operations further comprising sending the second data to a user device associated with the user, the second data causing display of the recommendation on the user device, wherein the recommendation, when displayed on the user device, includes an indication of why the recommendation is being surfaced on the user device.

15. The system of claim 11 , the operations further comprising sending the second data to a user device associated with the user, the second data causing display of the recommendation on the user device, wherein the recommendation, when displayed on the user device, includes an option that, when selected, causes the action to be performed.

16. The system of claim 11 , the operations further comprising:

sending the second data to a user device associated with the user, the second data causing display of the recommendation on the user device, wherein the recommendation, when displayed on the user device, includes an option that, when selected, indicates subsequent recommendations are not to be sent;

receiving user input data indicating selection of the option; and

including an indication of the user input data in the use data such that the subsequent recommendations are not sent to the user device.

17. The system of claim 11 , wherein:

the use data indicates that the past instances occurred during a period of time;

the first data indicates that the directive was sent during the period of time; and

determining that the user is interacting with the voice interface device occurs during the period of time.

18. The system of claim 11 , the operations further comprising:

determining that the user account data indicates occasional repeat actions are enabled; and

wherein causing the recommendation to be output comprises causing the recommendation to be output based at least in part on the user account data indicating the occasional repeat actions are enabled.

19. The system of claim 11 , the operations further comprising:

receiving audio data from the voice interface device;

determining, from the audio data, that the user is in proximity to the voice interface device; and

wherein determining that the user is interacting with the voice interface device comprises determining that the user is interacting with the voice interface device based at least in part on the user being in proximity to the voice interface device.

20. The system of claim 11 , the operations further comprising:

determining that the voice interface device is utilized to control operation of the accessory device more than other voice interface devices associated with the user account data; and

selecting, based at least in part on the voice interface device being utilized to control operation of the accessory device more than the other voice interface devices, the accessory device to perform the action in response to receiving user input data accepting the recommendation.

Assignments (1)

Continuity (1)

References Cited (63)

Cited By (1)

US 12,482,465