IP Library › Granted Patent US 11,823,675
Granted Patent B2
US 11,823,675 · App. 17/471,695 · Granted Nov 21, 2023

Display mode dependent response generation with latency considerations

Inventor: Jian Wei Leong (San Francisco, CA)
Assignee: GOOGLE LLC
G10L15/22G06F3/165G06F3/167G10L15/1822G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,823,675
App. No.
17/471,695
Granted
Nov 21, 2023
Kind
B2
Abstract

The present disclosure is generally related to a data processing system to process data packets in a voice activated computer network environment. The data processing system can improve the efficiency of the network by generating non-video data responses to voice commands received from a client device if a display associated with a client device is in an OFF state. A digital assistant application executed on the client device can send to the data processing system client device configuration data, which includes the state of the display device, among status data of other components of the client device. The data processing system can receive a current volume of speakers associated with the client device, and set a volume level for the client device based on the current volume level and a minimum response volume level at the client device.

Claims (82)

1. A system to process packetized audio signals in a voice activated computer network, comprising:

a digital assistant application executed by a processor of a client device to transmit data packets via a client device network interface over a network, the data packets including a first input audio signal, detected by a sensor of the client device, and first client device configuration data;

a natural language processor component executed by a data processing system to receive, over the network, via a network interface of the data processing system, the data packets, the natural language processor component to parse the first input audio signal to identify a first request and a first trigger keyword corresponding to the first request; and

a client device configuration appliance executed by the data processing system to, in response to receiving the data packets:

determine, from the first client device configuration data, a current volume level of at least one speaker at the client device,

generate, based on the current volume level, first client device configuration settings including a first volume level for the at least one speaker at the client device, the generated first volume level being the greater of the current volume level and a minimum response audio level, and

generate, based in part on the first trigger keyword, a first response data structure, the first response data structure including audio data, and

transmit, over the network, via the network interface of the data processing system and to the digital assistant application at the client device:

the first client device configuration settings, and

the first response data structure.

2. The system of claim 1 , comprising:

the client device configuration appliance executed by the data processing system to:

generate the first volume level in response to determining that the first response data structure includes audio data.

3. The system of claim 1 , comprising:

the client device configuration appliance executed by the data processing system to:

generate status video data, and

transmit, over the network, via the network interface of the data processing system, the status video data to the digital assistant application at the client device prior to the transmission of the response data structure,

the digital assistant application executed by the processor of the client device to display the status video on the display associated with the client device.

4. The system of claim 1 , comprising:

the client device configuration appliance executed by the data processing system to:

generate configuration settings indicating activation of light emitting status indicators on the client device, and

transmit, over the network, via the network interface of the data processing system, the configuration settings to the digital assistant application at the client device prior to the transmission of the first response data structure.

5. The system of claim 4 , comprising:

the digital assistant application executed by a processor of a client device to:

receive, via the network interface of the client device, over the network, the configuration settings indicating activation of light emitting status indicators on the client device, and

cause light emitting status indicators on the client device to activate.

6. The system of claim 1 , wherein the client device includes a radio-frequency tuner.

7. The system of claim 1 , comprising:

the digital assistant application executed by a processor of a client device to:

receive, via the network interface of the client device, over the network, the first response data structure responsive to the first request,

determine the audio data of the first response data structure, and

cause the audio data to be played, at the first volume level, on at least one speaker of the client device.

8. A method to process packetized audio signals in a voice

activated computer network, comprising:

receiving, via a network interface, data packets transmitted by a digital assistant application executed by a processor of a client device, the data packets including a first input audio signal, detected by a sensor of the client device, and a first client device configuration data;

parsing the first input audio signal to identify a first request and a first trigger keyword corresponding to the first request;

in response to receiving and parsing the data packets:

determining, from the first client device configuration data, a current volume level of at least one speaker at the client device,

generating, based on the current volume level, first client device configuration settings including a first volume level for the at least one speaker at the client device, wherein generating the first client device configuration settings comprises generating the first volume level as the greater of the current volume level and a minimum response audio level, and

generating, based in part on the first trigger keyword, a first response data structure, the first response data structure including audio data, and

transmitting, over the network and to the digital assistant application at the client device:

the first client device configuration settings, and

the first response data structure.

9. The method of claim 8 , wherein generating the first volume level is in response to determining that the first response data structure includes audio data.

10. The method of claim 8 , further comprising:

generating status video data, and

transmitting, over the network and via the network interface, the status video data to the digital assistant application at the client device prior to the transmission of the response data structure, the digital assistant application at the client device to display the status video on the display associated with the client device.

11. The method of claim 8 , further comprising:

generating configuration settings indicating activation of light emitting status indicators on the client device, and

transmitting, over the network and via the network interface, the configuration settings to the digital assistant application at the client device prior to the transmission of the first response data structure.

12. The method of claim 11 , further comprising:

receiving, at the client device and over the network, the configuration settings indicating activation of light emitting status indicators on the client device, and

causing, by the client device, light emitting status indicators on the client device to activate.

13. The method of claim 8 , wherein the client device includes a radio-frequency tuner.

14. The method of claim 8 , further comprising:

receiving, at the client device and over the network, the first response data structure responsive to the first request,

determining, at the client device, the audio data of the first response data structure, and

causing, at the client device, the audio data to be played, at the first volume level, on at least one speaker of the client device.

15. A non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by a processor, cause the processor to:

receive, via a network interface, data packets transmitted by a digital assistant application executed by a processor of a client device, the data packets including a first input audio signal, detected by a sensor of the client device, and a first client device configuration data;

parse the first input audio signal to identify a first request and a first trigger keyword corresponding to the first request; and

in response to receiving and parsing the data packets:

determine, from the first client device configuration data, a current volume level of at least one speaker at the client device,

generate, based on the current volume level, first client device configuration settings including a first volume level for the at least one speaker at the client device, wherein generating the first client device configuration settings comprises generating the first volume level as the greater of the current volume level and a minimum response audio level, and

generate, based in part on the first trigger keyword, a first response data structure, the first response data structure including audio data, and

transmit, over the network and to the digital assistant application at the client device:

the first client device configuration settings, and

the first response data structure.

16. The non-transitory computer-readable medium of claim 15 , wherein generating the first volume level is in response to determining that the first response data structure includes audio data.

17. The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the processor to:

generate status video data, and

transmit, over the network and via the network interface, the status video data to the digital assistant application at the client device prior to the transmission of the response data structure, the digital assistant application at the client device to display the status video on the display associated with the client device.

18. The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the processor to:

generate configuration settings indicating activation of light emitting status indicators on the client device, and

transmit, over the network and via the network interface, the configuration settings to the digital assistant application at the client device prior to the transmission of the first response data structure.

19. The non-transitory computer-readable medium of claim 18 , wherein the instructions further cause the processor to:

receive, at the client device and over the network, the configuration settings indicating activation of light emitting status indicators on the client device, and

cause, by the client device, light emitting status indicators on the client device to activate.

20. The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the processor to:

receive, at the client device and over the network, the first response data structure responsive to the first request,

determine, at the client device, the audio data of the first response data structure, and

cause, at the client device, the audio data to be played, at the first volume level, on at least one speaker of the client device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2021
From: LEONG, JIAN WEI
To: GOOGLE LLC
Reel/Frame 057566/0807 →
Continuity (2)
Continuation 16461424
Related Publication 20210407509A1 · Dec 30, 2021
Cited By (1)
US 12,243,527