IP Library › Granted Patent US 10,339,769
Granted Patent B2
US 10,339,769 · App. 15/815,646 · Granted Jul 2, 2019

Server-provided visual output at a voice interface device

Inventors: Kenneth Mixter (Los Altos Hills, CA); Yuan Yuan (Redwood City, CA)
Assignee: GOOGLE LLC
G08B5/36G06F3/167G08B3/10G09G3/14G10L15/22H04L12/282H04L12/2825G10L2015/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,339,769
App. No.
15/815,646
Granted
Jul 2, 2019
Kind
B2
Abstract

A method at an electronic device with an array of indicator lights includes: obtaining first visual output instructions stored at the electronic device, where the first visual output instructions control operation of the array of indicator lights based on operating state of the electronic device; receiving a voice input; obtaining from a remote system a response to the voice input and second visual output instructions, where the second visual output instructions are provided by the remote system along with the response in accordance with a determination that the voice input satisfies one or more criteria; executing the response; and displaying visual output on the array of indicator lights in accordance with the second visual output instructions, where otherwise in absence of the second visual output instructions the electronic device displays visual output on the array of indicator lights in accordance with the first visual output instructions.

Claims (59)

1. A method, comprising:

at an electronic device with one or more microphones, a speaker, an array of indicator lights, one or more processors, and memory storing one or more programs for execution by the one or more processors:

obtaining first visual output instructions stored at the electronic device, wherein the first visual output instructions control operation of the array of indicator lights based on operating state of the electronic device, including operating states of applications executing thereon;

receiving a voice input;

obtaining from a remote system a response to the voice input and second visual output instructions, wherein the second visual output instructions are provided by the remote system along with the response in accordance with a determination that the voice input satisfies one or more criteria;

executing the response;

in response to obtaining the second visual output instructions, changing operation of the array of indicator lights to respond to the second visual output instructions instead of the first visual output instructions; and

in response to a determination of an absence of the second visual output instructions, displaying visual output on the array of indicator lights in accordance with the first visual output instructions.

2. The method of claim 1 , wherein displaying the visual output on the array of indicator lights in accordance with the second visual output instructions is concurrent with executing the response.

3. The method of claim 1 , further comprising:

obtaining from the remote system audible output instructions associated with the second visual output instructions; and

outputting audible output on the speaker in accordance with the audible output instructions;

wherein the audible output instructions are supplemental to an audible output associated with the response to the voice input.

4. The method of claim 1 , wherein the one or more criteria comprise: the voice input including a request associated with a holiday specified in the voice input.

5. The method of claim 1 , wherein the one or more criteria comprise: a date on which the voice input is received corresponds to a holiday.

6. The method of claim 1 , wherein the one or more criteria comprise: the voice input is associated with an on-going game session at the electronic device.

7. The method of claim 1 , wherein the array of indicator lights comprises an array of LED lights.

8. The method of claim 1 , wherein the array of indicator lights comprises full-color lights.

9. A method, comprising:

at an electronic device with one or more microphones, a speaker, an array of indicator lights, one or more processors, and memory storing one or more programs for execution by the one or more processors:

obtaining first visual output instructions stored at the electronic device, wherein the first visual output instructions control operation of the array of indicator lights based on operating state of the electronic device, including operating states of applications executing thereon;

obtaining from a remote system second visual output instructions;

performing an operation;

in association with performing the operation, determining whether one or more alternate visual output criteria are satisfied;

in accordance with a determination that the alternate visual output criteria are satisfied, changing operation of the array of indicator lights to respond to the second visual output instructions instead of the first visual output instructions; and

in accordance with the determination that the alternate visual output criteria are not satisfied, displaying visual output on the array of indicator lights in accordance with the first visual output instructions.

10. The method of claim 9 , wherein displaying the visual output on the array of indicator lights in accordance with the second visual output instructions is concurrent with performing the operation.

11. The method of claim 9 , further comprising:

obtaining from the remote system audible output instructions associated with the second visual output instructions; and

outputting audible output on the speaker in accordance with the audible output instructions;

wherein the audible output instructions are supplemental to an audible output associated with performing the operation.

12. The method of claim 9 , wherein the one or more alternate visual output criteria comprise: the operation includes playing a response associated with a holiday.

13. The method of claim 9 , wherein the one or more alternate visual output criteria comprise: the operation is performed on a date that corresponds to a holiday.

14. The method of claim 9 , wherein the one or more criteria comprise: the operation is associated with an on-going game session at the electronic device.

15. The method of claim 9 , wherein the electronic device is one of a group of communicatively coupled devices, further comprising, when the operation is directed to the group of devices and is determined by the electronic device to satisfy the alternate visual output criteria, causing each of the group of devices to perform the operation and display light patterns in accordance with the second visual output instructions.

16. An electronic device, comprising:

one or more microphones;

a speaker;

an array of indicator lights;

one or more processors; and

memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for performing:

obtaining first visual output instructions stored at the electronic device, wherein the first visual output instructions control operation of the array of indicator lights based on operating state of the electronic device, including operating states of applications executing thereon;

receiving a voice input;

obtaining from a remote system a response to the voice input and second visual output instructions, wherein the second visual output instructions are provided by the remote system along with the response in accordance with a determination that the voice input satisfies one or more criteria;

executing the response;

in response to obtaining the second visual output instructions, changing operation of the array of indicator lights to respond to the second visual output instructions instead of the first visual output instructions; and

in response to an absence of the second visual output instructions, displaying visual output on the array of indicator lights in accordance with the first visual output instructions.

17. The electronic device of claim 16 , wherein displaying the visual output on the array of indicator lights in accordance with the second visual output instructions is concurrent with executing the response.

18. The electronic device of claim 17 , wherein the instructions further comprise:

obtaining from the remote system audible output instructions associated with the second visual output instructions; and

outputting audible output on the speaker in accordance with the audible output instructions;

wherein the audible output instructions are supplemental to an audible output associated with the response to the voice input.

19. The electronic device of claim 17 , wherein the instructions further comprise:

obtaining from a remote system second visual output instructions;

performing an operation;

in association with performing the operation, determining whether one or more alternate visual output criteria are satisfied;

in accordance with a determination that the alternate visual output criteria are satisfied, displaying visual output on the array of indicator lights in accordance with the second visual output instructions; and

in accordance with the determination that the alternate visual output criteria are not satisfied, displaying visual output on the array of indicator lights in accordance with the first visual output instructions.

20. The electronic device of claim 19 , wherein the electronic device is one of a group of communicatively coupled devices, further comprising, when the operation is directed to the group of devices and is determined by the electronic device to satisfy the alternate visual output criteria, the instructions further comprise: causing each of the group of devices to perform the operation and display light patterns in accordance with the second visual output instructions.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 17, 2018
From: MIXTER, KENNETH; YUAN, YUAN
To: GOOGLE LLC
Reel/Frame 044640/0058 →
Continuity (2)
Provisional Application 62424354 · Nov 18, 2016
Related Publication 20180144590A1 · May 24, 2018
Cited By (29)
US 12,197,699 US 12,210,730 US 12,223,228 US 12,242,702 US 12,244,755 US 12,256,128 US 12,260,059 US 12,262,089 US 12,265,364 US 12,265,696 US 12,267,622 US 12,301,979 US 12,302,035 US 12,348,663 US 12,368,946 US 12,379,827 US 12,381,924 US 12,422,976 US 12,449,961 US 12,452,389 US 12,504,944 US 12,526,361 US 12,541,338 US 12,563,299 US 12,578,837 US 12,615,491 US 12,620,155 US 12,732,665 US 12,737,049