IP Library Granted Patent US 10,893,088
Granted Patent B2
US 10,893,088 · App. 16/818,736 · Granted Jan 12, 2021

Sequence dependent data message consolidation in a voice activated computer network environment

Inventors: Gaurav Bhaya (Sunnyvale, CA); Robert James Stets, Jr. (Mountain View, CA)
Assignee: Google LLC
H04L67/02G06F16/90332G06F16/951G06F40/279G10L15/26H04L67/10H04L67/327
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,893,088
App. No.
16/818,736
Granted
Jan 12, 2021
Kind
B2
Abstract

Systems and methods to combine multiple voice activated audio input data packets that indicate sequence dependent operations are provided. A natural language processor component can receive first and second input audio signal from a client computing device, and can identify respective requests and corresponding trigger keywords. A direct action application programming interface (“API”) can generate respective action data structures, and can construct respective data transmissions including the respective action data structures. A thread optimization component can obtain data packets of the first data transmission, and can obtain data packets of the second data transmission. The thread optimization component can determine, based on a heuristic technique applied to the data packets of the respective data transmissions a sequence dependency parameter. The thread optimization component can merge, based on a comparison of the sequence dependency parameter with a threshold, the first and second data transmissions into a single thread.

Claims (68)

1. A system to perform sequence dependent operations in a voice-based computing environment, comprising:

a natural language processor component executed by a data processing system to receive, via an interface of the data processing system, data packets comprising one or more input audio signals detected by a sensor of a client computing device;

the natural language processor component to parse the one or more input audio signals to identify one or more requests and one or more trigger keywords corresponding to the one or more requests;

a direct action application programming interface (“API”) of the data processing system to generate, based on the one or more trigger keywords and in response to the one or more requests, a first action data structure for a first action and a second action data structure for a second action; and

a thread optimization component executed by the data processing system to:

obtain data packets of the first action data structure, and to obtain data packets of the second action data structure;

determine, based on a heuristic technique applied to the data packets of the first action data structure and the data packets of the second action data structure, a sequence dependency parameter;

combine, based on a comparison of the sequence dependency parameter with a threshold, the data packets of the first action data structure and the data packets of the second action data structure into a single thread; and

identify a series of sequence dependent actions for the single thread;

a content selector component to select a content item for at least one of the series of sequence dependent actions; and

the data processing system to:

provide the content item to the client computing device;

identify a condition associated with the second action of the second action data structure based on a time associated with the first action of the first action data structure;

receive an indication of termination of the condition; and

transmit the data packets of the second action data structure to a service provider computing device subsequent to the termination of the condition.

2. The system of claim 1 , comprising the data processing system to:

generate the first action data structure based on a first trigger keyword of the one or more trigger keywords; and

construct a first data transmission including the first action data structure to establish a first communication session between a first service provider computing device and the client computing device, wherein data packets of the first communication session routed from the client computing device to the first service provider computing device via the data processing system; and

including the first data transmission in the single thread.

3. The system of claim 1 , comprising the data processing system to:

generate the first action data structure based on a first trigger keyword of the one or more trigger keywords, and the second action data structure based on a second trigger keyword of the one or more trigger keywords.

4. The system of claim 1 , comprising the data processing system to:

construct a first data transmission including the first action data structure to establish a first communication session between a first service provider computing device and the client computing device;

construct a second data transmission including the second action data structure to establish a second communication session between a second service provider computing device and the client computing device; and

merge, based on the comparison of the sequence dependency parameter with the threshold, the first data transmission with the second data transmission into the single thread.

5. The system of claim 1 , comprising the data processing system to:

detect a completion of the first action of the first action data structure; and

transmit the data packets of the single thread to the service provider computing device subsequent to the completion of the first action, wherein the single thread indicates that the first action is scheduled to occur prior to the second action of the second action data structure.

6. The system of claim 1 , wherein the sequence dependency parameter indicates that the first action corresponding to the first action data structure is scheduled to occur prior to the second action corresponding to the second action data structure.

7. The system of claim 1 , comprising the data processing system to:

to identify a second condition associated with the second action corresponding to the second action data structure based on the time associated with the first action corresponding to the first action data structure; and

prevent transmission of the data packets of the second action data structure to the service provider computing device responsive to the second condition.

8. The system of claim 1 , wherein the single thread indicates a series of sequence dependent actions for the single thread, the series of sequence dependent actions including the first action, the second action subsequent to the first action, and a third action subsequent to the second action.

9. The system of claim 1 , comprising:

the data processing system to identify a content item for the second action data structure prior to execution of the first action data structure.

10. A method of performing sequence dependent operations in a voice-based computing environment, comprising:

receiving, by a data processing system via an interface, data packets comprising one or more input audio signals detected by a sensor of a client computing device;

parsing, by the data processing system, the one or more input audio signals to identify one or more requests and one or more trigger keywords corresponding to the one or more requests;

generating, by the data processing system, based on the one or more trigger keywords and in response to the one or more requests, a first action data structure for a first action and a second action data structure for a second action;

obtaining, by the data processing system, data packets of the first action data structure, and to obtain data packets of the second action data structure;

determining, by the data processing system, based on a heuristic technique applied to the data packets of the first action data structure and the data packets of the second action data structure, a sequence dependency parameter;

combining, by the data processing system, based on a comparison of the sequence dependency parameter with a threshold, the data packets of the first action data structure and the data packets of the second action data structure into a single thread;

identifying, by the data processing system, a series of sequence dependent actions for the single thread;

selecting, by the data processing system, a content item for at least one of the series of sequence dependent actions;

providing, by the data processing system, the content item to the client computing device;

identifying, by the data processing system, a condition associated with the second action of the second action data structure based on a time associated with the first action of the first action data structure;

receiving, by the data processing system, an indication of termination of the condition; and

transmitting, by the data processing system, the data packets of the second action data structure to a service provider computing device subsequent to the termination of the condition.

11. The method of claim 10 , comprising:

generating the first action data structure based on a first trigger keyword of the one or more trigger keywords;

constructing a first data transmission including the first action data structure to establish a first communication session between a first service provider computing device and the client computing device, wherein data packets of the first communication session routed from the client computing device to the first service provider computing device via the data processing system; and

providing the first data transmission in the single thread.

12. The method of claim 10 , comprising:

generating the first action data structure based on a first trigger keyword of the one or more trigger keywords, and the second action data structure based on a second trigger keyword of the one or more trigger keywords.

13. The method of claim 10 , comprising:

constructing a first data transmission including the first action data structure to establish a first communication session between a first service provider computing device and the client computing device;

constructing a second data transmission including the second action data structure to establish a second communication session between a second service provider computing device and the client computing device; and

combining, based on the comparison of the sequence dependency parameter with the threshold, the first data transmission with the second data transmission into the single thread.

14. The method of claim 10 , comprising:

detecting a completion of the first action of the first action data structure; and

transmitting the data packets of the single thread to the service provider computing device subsequent to the completion of the first action, wherein the single thread indicates that the first action is scheduled to occur prior to the second action.

15. The method of claim 10 , wherein the sequence dependency parameter indicates that the first action corresponding to the first action data structure is scheduled to occur prior to the second action corresponding to the second action data structure.

16. The method of claim 10 , comprising:

identifying a second condition associated with the second action corresponding to the second action data structure based on the time associated with the first action corresponding to the first action data structure; and

preventing transmission of the data packets of the second action data structure to the service provider computing device responsive to the second condition.

17. The method of claim 10 , wherein the single thread indicates a series of sequence dependent actions for the single thread, the series of sequence dependent actions including the first action, the second action subsequent to the first action, and a third action subsequent to the second action.

18. The method of claim 10 , comprising:

identifying a content item for the second action data structure prior to execution of the first action data structure.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 16, 2020
From: BHAYA, GAURAV; STETS, ROBERT JAMES, JR.
To: GOOGLE INC.
Reel/Frame 052123/0086 →
CHANGE OF NAME Recorded Mar 16, 2020
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 052171/0447 →
Continuity (2)
Continuation 15395725 · Dec 30, 2016
Related Publication 20200213382A1 · Jul 2, 2020
Cited By (2)
US 12,501,211 US 12,542,997