IP Library › Granted Patent US 12,230,270
Granted Patent B1
US 12,230,270 · App. 18/471,065 · Granted Feb 18, 2025

Voice-controlled device switching between modes based on speech input

Inventors: Valere Joseph Vanderschaegen (Kalama, WA); Kazim Das (San Jose, CA); Donald L. Cantrell (Oakland, CA); Johan Le Nerriec (Milpitas, CA); Joseph Pedro Tavares (Kenmore, WA)
Assignee: Amazon Technologies, Inc.
G10L15/22H04N21/42203H04N21/47202H04N21/47217G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,230,270
App. No.
18/471,065
Granted
Feb 18, 2025
Kind
B1
Abstract

Techniques for presenting content by a voice-controlled device are described. In an example, the voice-controlled device is operatively coupled to a presentation device and supports dual mode functionalities. In a first mode, the voice-controlled device sends content for presentation at the presentation device. In a second mode, the voice-controlled device presents the content at a presentation interface of the voice-controlled device. Based on speech input from a user indicating an issue with a content presentation in the first mode, the voice-controlled device switches to the second mode and presents a message at the presentation interface indicating that subsequent content presentations would be presented at this interface. The voice-controlled device remains in the second mode until receiving additional speech input necessitating a switch to the first mode.

Claims (97)

1. A computer-implemented method comprising:

receiving, by an input device, first speech input, the input device being operated in a first mode and coupled to a first port of an output device, the first speech input requesting at least video content;

determining, by the input device, that a video response is to be presented at the output device based at least in part on the first speech input;

causing, by the input device in the first mode, an audio message to be presented over a speaker, the audio message indicating that the video response will be presented at the output device;

switching, by the input device, to a second mode from the first mode by at least sending a control signal to the output device, the control signal associated with changing an active input port of the output device to the first port; and

sending, by the input device in the second mode, the video response to the output device over the first port.

2. The computer-implemented method of claim 1 , further comprising:

receiving, by the input device, second speech input while the input device is in the second mode, the second speech input indicating an issue with a presentation of the video response;

switching, by the input device, to the first mode based at least in part on the second speech input;

resetting, by the input device, a volume level of the speaker in the first mode; and

causing, by the input device in the first mode, an audio response to be presented over the speaker according to the volume level.

3. The computer-implemented method of claim 2 , further comprising:

receiving, by the input device, third speech input;

determining, by the input device, that audio only is to be presented based at least in part on the third speech input; and

causing, by the input device in the first mode, a second audio response to be presented over the speaker in response to the third speech input.

4. An input device comprising:

a data interface configured to operatively couple the input device with an output device;

a microphone;

a speaker;

a processor; and

a memory storing computer-readable instructions that, upon execution by the processor, configure the input device to:

receive, at the microphone, first speech input while the input device is in a first mode, the first mode associated with presenting speech input responses at the speaker;

determine that a first response to the first speech input is to be presented at the output device;

present a first message over the speaker, the first message indicating that the first response will be presented at the output device;

switch, by the input device, to a second mode from the first mode, the second mode associated with presenting the first response at the output device; and

send, by the input device in the second mode, the first response to the output device over the data interface.

5. The input device of claim 4 , wherein the data interface comprises a high-definition multimedia interface (HDMI) interface, and wherein the memory stores additional computer-readable instructions that, upon execution by the processor, configure the input device to:

send a control signal to the output device, the control signal associated with changing an active input port of the output device to a first port of the output device coupled to the HDMI interface.

6. The input device of claim 5 , wherein the memory stores additional computer-readable instructions that, upon execution by the processor, configure the input device to:

receive, at the microphone, second speech input while the input device is in the second mode, the second speech input indicating an issue with a presentation of the first response;

switch to the first mode based at least in part on the second speech input; and

present, at the speaker, a second response to the second speech input.

7. The input device of claim 5 , wherein the memory stores additional computer-readable instructions that, upon execution by the processor, configure the input device to:

receive, while the input device is in the second mode, second speech input;

determine that a second response to the second speech input is to be presented at the output device;

determine that an activity state associated with the data interface is inactive;

send a control signal to the output device, the control signal associated with changing the activity state to active; and

send the second response to the output device.

8. The input device of claim 4 , wherein the computer-readable instructions further configure the input device to:

receive, while the input device is in the second mode, second speech input;

determine that a second response to the second speech input is to be presented at the output device;

determine that an activity state associated with the data interface is active;

cause a data stream to the output device to be paused; and

send the second response to the output device, wherein the second response is presented while the data stream is paused.

9. The input device of claim 4 , wherein the memory stores additional computer-readable instructions that, upon execution by the processor, configure the input device to:

receive, while the input device is in the second mode, second speech input;

determine that a second response to the second speech input is to be presented at the output device;

determine that an activity state associated with the data interface is active;

determine that a power state associated with the output device is unknown; and

send the second response to the output device.

10. The input device of claim 4 , wherein the computer-readable instructions further configure the input device to:

receive, while the input device is in the second mode, second speech input;

determine that a second response to the second speech input is to be presented at the output device;

determine that an activity state associated with the data interface is inactive;

determine that a power state associated with the output device is unknown;

send one or more control signals to the output device, the one or more control signals associated with changing the activity state to active and the power state to ON; and

send the second response to the output device.

11. The input device of claim 4 , wherein the memory stores additional computer-readable instructions that, upon execution by the processor, configure the input device to:

receive, while the input device is in the second mode, second speech input;

determine that an audio response to the second speech input is to be presented;

determine that an activity state associated with the data interface is inactive; and

present, while the input device is in the second mode the audio response at the speaker.

12. The input device of claim 11 , wherein the memory stores additional computer-readable instructions that, upon execution by the processor, configure the input device to:

receive, while the input device is in the second mode, second speech input;

determine that an audio response to the second speech input is to be presented;

determine that a power state of the output device is unknown;

determine that an activity state associated with the data interface is active; and

send the audio response to the output device.

13. The input device of claim 11 , wherein the memory stores additional computer-readable instructions that, upon execution by the processor, configure the input device to:

determine an activity state associated with the data interface, wherein the data interface comprises a high-definition multimedia interface (HDMI) interface, wherein the activity state indicates that the HDMI interface is inactive; and

set the activity state to active based at least in part on an active hot plug detect (HPD) connection and an active high-bandwidth digital content protection (HDCP) connection associated with a first port of the output device.

14. The input device of claim 4 , wherein the memory stores additional computer-readable instructions that, upon execution by the processor, configure the input device to:

receive, while the input device is in the second mode, second speech input;

determine that an audio response to the second speech input is to be presented;

determine that an activity state associated with the data interface is active;

cause a data stream to the output device to be paused; and

send the audio response to the output device wherein the audio response is presented while the data stream is paused.

15. One or more non-transitory computer-readable storage media storing instructions that, upon execution on an input device, cause the input device to perform operations comprising:

receiving, by the input device, first speech input while the input device is in a first mode, the first mode associated with presenting speech input responses at a speaker;

determining that a first response to the first speech input is to be presented at an output device, the input device communicatively coupled with the output device over a data interface;

causing a first message to be presented at the speaker, the first message indicating that the first response will be presented at the output device;

switching, by the input device, to a second mode from the first mode, the second mode associated with presenting the first response at the output device; and

sending, by the input device in the second mode, the first response to the output device over the data interface.

16. The one or more non-transitory computer-readable storage media of claim 15 further comprising additional instructions that, upon execution on the input device, cause the input device to perform additional operations comprising:

receiving, while the input device is in the first mode, second speech input;

determining that a second response to the second speech input is to be presented at the speaker; and

causing the second response to be presented at the speaker while remaining in the first mode.

17. The one or more non-transitory computer-readable storage media of claim 15 further comprising additional instructions that, upon execution on the input device, cause the input device to perform additional operations comprising:

receiving, while the input device is in the first mode, second speech input;

determining at least one of a power state or an activity state of the output device;

determining that an audio response to the second speech input is to be presented; and

presenting the audio response at the speaker while remaining in the first mode and without controlling the power state of the activity state of the output device.

18. The one or more non-transitory computer-readable storage media of claim 15 further comprising additional instructions that, upon execution on the input device, cause the input device to perform additional operations comprising:

determining at least one of a power state or an activity state of the output device; and

sending, to the output device, one or more control signals based at least in part on the power state or the activity state, the one or more control signals associated with a presentation of the first response at the output device.

19. The one or more non-transitory computer-readable storage media of claim 18 , wherein the one or more control signals comprise a signal to turn on power to the output device, wherein the signal is based at least in part on the power state indicating that the output device is powered off.

20. The one or more non-transitory computer-readable storage media of claim 18 , wherein the one or more control signals comprise a signal to set an active data interface of the output device to a port operatively connected to the data interface of the input device, wherein the signal is based at least in part on the activity state indicating that the active data interface of the output device is set to a different port.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2023
From: VANDERSCHAEGEN, VALERE JOSEPH; DAS, KAZIM; CANTRELL, DONALD L.; LE NERRIEC, JOHAN; TAVARES, JOSEPH PEDRO
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 064973/0640 →
Continuity (2)
Continuation 17222282 · Apr 5, 2021
Continuation 16143840 · Sep 27, 2018
References Cited (14)
US 10560737B2 · Lawrence et al. · 2020 [cited by applicant]
US 10847176B2 · Kristjansson et al. · 2020 [cited by applicant]
US 10978062B1 · Vanderschaegen · 2021 [cited by examiner]
US 11810560B1 · Vanderschaegen · 2023 [cited by examiner]
US 20040046677A1 · Dresti · 2004 [cited by examiner]
US 20150373393A1 · Lee et al. · 2015 [cited by applicant]
US 20170083285A1 · Meyers et al. · 2017 [cited by applicant]
US 20180211665A1 · Park et al. · 2018 [cited by applicant]
US 20180330069A1 · Quinn et al. · 2018 [cited by applicant]
US 20190311718A1 · Huber et al. · 2019 [cited by applicant]
US 20200057607A1 · Jeong · 2020 [cited by examiner]
U.S. Appl. No. 16/143,840, “Notice of Allowance,” mailed Dec. 11, 2020, 20 pages. [cited by applicant]
U.S. Appl. No. 17/222,282, “Non-Final Office Action,” mailed Mar. 16, 2023, 11 pages. [cited by applicant]
U.S. Appl. No. 17/222,282, “Notice of Allowance,” mailed Jun. 23, 2023, 8 pages. [cited by applicant]