IP Library › Granted Patent US 11,436,411
Granted Patent B2
US 11,436,411 · App. 16/698,350 · Granted Sep 6, 2022

Detecting continuing conversations with computing devices

Inventors: Nathan David Howard (Mountain View, CA); Gabor Simko (Santa Clara, CA); Andrei Giurgiu (Zurich, CH); Behshad Behzadi (Oechsli, CH); Marcin M. Nowak-Przygodzki (Zurich, CH)
Assignee: GOOGLE LLC
G06F40/284G06F16/9024G06F16/90335G06N5/02G10L15/08G10L15/22G10L25/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,436,411
App. No.
16/698,350
Granted
Sep 6, 2022
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for detecting a continued conversation are disclosed. In one aspect, a method includes the actions of receiving first audio data of a first utterance. The actions further include obtaining a first transcription of the first utterance. The actions further include receiving second audio data of a second utterance. The actions further include obtaining a second transcription of the second utterance. The actions further include determining whether the second utterance includes a query directed to a query processing system based on analysis of the second transcription and the first transcription or a response to the first query. The actions further include configuring the data routing component to provide the second transcription of the second utterance to the query processing system as a second query or bypass routing the second transcription.

Claims (72)

1. A computer-implemented method comprising:

receiving, by a computing device, first audio data of a first utterance;

obtaining, by the computing device, a first transcription of the first utterance;

receiving, by the computing device, second audio data of a second utterance;

obtaining, by the computing device, a second transcription of the second utterance;

determining, by the computing device, whether the second utterance includes a query directed to a query processing system based on analysis of (i) the second transcription and (ii) the first transcription or a response to the first utterance, wherein determining whether the second utterance includes a query directed to the query processing system comprises:

tokenizing the second transcription,

determining whether a pronoun, in the second transcription, refers to a noun in the first transcription or in the response to the first utterance, and

determining whether the second utterance includes a query directed to the query processing system based on whether the pronoun, in the second transcription, refers to the noun in the first transcription or in the response to the first utterance; and

based on determining whether the second utterance includes a query directed to the query processing system, configuring, by the computing device, a data routing component to (i) provide the second transcription of the second utterance to the query processing system as a second query or (ii) bypass routing the second transcription so that the second transcription is not provided to the query processing system.

2. The method of claim 1 , wherein determining whether the second utterance includes a query directed to the query processing system is based on analysis of (i) the second transcription and (ii) the first transcription.

3. The method of claim 1 , wherein determining whether the second utterance includes a query directed to the query processing system is based on analysis of (i) the second transcription and (ii) the response to the first utterance.

4. The method of claim 1 , wherein determining whether the second utterance includes a query directed to the query processing system is based on analysis of the second transcription, the first transcription, and the response to the first utterance.

5. The method of claim 1 , wherein:

determining whether the second utterance includes a query directed to the query processing system comprises determining that the second utterance includes a query directed to the query processing system, and

configuring the data routing component to provide the second transcription of the second utterance to the query processing system as the second query.

6. The method of claim 1 , wherein:

determining whether the second utterance includes a query directed to the query processing system comprises determining that the second utterance does not include a query directed to the query processing system, and

configuring the data routing component to bypass routing the second transcription so that the second transcription is not provided to the query processing system.

7. The method of claim 1 , wherein:

determining whether the second utterance includes a query directed to the query processing system comprises:

tokenizing (i) the second transcription and (ii) the first transcription or the response to the first utterance; and

comparing (i) terms of the second transcription and (ii) terms of the first transcription or the response to the first utterance.

8. The method of claim 7 , wherein comparing (i) the terms of the second transcription and (ii) the terms of the first transcription or the response to the first utterance comprises determining a relationship between (i) the terms of the second transcription and (ii) the terms of the first transcription or the response to the first utterance in a knowledge graph.

9. The method of claim 1 , wherein:

determining whether the second utterance includes a query directed to the query processing system is based on comparing (i) a grammatical structure of the second transcription and (ii) a grammatical structure of the first transcription or the response to the first utterance.

10. The method of claim 1 , comprising:

determining content on a user interface; and

determining whether the second utterance includes a query directed to the query processing system based on the content of the user interface.

11. The method of claim 1 , comprising:

determining a location of a user device that detected the first utterance and the second utterance through a microphone; and

determining whether the second utterance includes a query directed to the query processing system based on the location of the user device that detected the first utterance and the second utterance through the microphone.

12. The method of claim 1 , comprising:

determining a time that the computing device receives the second audio data of the second utterance; and

determining whether the second utterance includes a query directed to the query processing system based on the time that the computing device receives the second audio data of the second utterance.

13. The method of claim 1 , wherein:

analyzing (i) the second transcription and (ii) the first transcription or the response to the first utterance comprises comparing the second transcription with one or more queries in a query log, and

determining whether the second utterance includes a query directed to the query processing system is based on comparing the second transcription with the one or more queries in the query log.

14. The method of claim 1 , comprising:

providing, by the data routing component of the computing device, the first transcription of the first utterance as a first query to the query processing system;

receiving, from the query processing system, a response to the first query; and

providing, for output by the computing device, the response to the first query.

15. A system comprising:

one or more computers; and

one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform the operations comprising:

receiving, by a computing device, first audio data of a first utterance;

obtaining, by the computing device, a first transcription of the first utterance;

receiving, by the computing device, second audio data of a second utterance;

obtaining, by the computing device, a second transcription of the second utterance;

determining, by the computing device, whether the second utterance includes a query directed to a query processing system based on analysis of (i) the second transcription and (ii) the first transcription or a response to the first utterance, wherein determining whether the second utterance includes a query directed to the query processing system comprises

tokenizing the second transcription,

determining whether a pronoun, in the second transcription, refers to a noun in the first transcription or in the response to the first utterance, and

determining whether the second utterance includes a query directed to the query processing system based on whether the pronoun, in the second transcription, refers to the noun in the first transcription or in the response to the first utterance; and

based on determining whether the second utterance includes a query directed to the query processing system, configuring, by the computing device, a data routing component to (i) provide the second transcription of the second utterance to the query processing system as a second query or (ii) bypass routing the second transcription so that the second transcription is not provided to the query processing system.

16. The system of claim 15 , wherein determining whether the second utterance includes a query directed to a query processing system is based on analysis of the second transcription, the first transcription, and the response to the first utterance.

17. The system of claim 15 , wherein:

determining whether the second utterance includes a query directed to the query processing system comprises determining that the second utterance does not include a query directed to the query processing system, and

configuring the data routing component to bypass routing the second transcription so that the second transcription is not provided to the query processing system.

18. The system of claim 15 , wherein:

determining whether the second utterance includes a query directed to the query processing system comprises:

tokenizing (i) the second transcription and (ii) the first transcription or the response to the first utterance; and

comparing (i) terms of the second transcription and (ii) terms of the first transcription or the response to the first utterance.

19. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform the operations comprising:

receiving, by a computing device, first audio data of a first utterance;

obtaining, by the computing device, a first transcription of the first utterance;

receiving, by the computing device, second audio data of a second utterance;

obtaining, by the computing device, a second transcription of the second utterance;

determining, by the computing device, whether the second utterance includes a query directed to a query processing system based on analysis of (i) the second transcription and (ii) the first transcription or a response to the first utterance, wherein determining whether the second utterance includes a query directed to the query processing system comprises

tokenizing the second transcription,

determining whether a pronoun, in the second transcription, refers to a noun in the first transcription or in the response to the first utterance, and

determining whether the second utterance includes a query directed to the query processing system based on whether the pronoun, in the second transcription, refers to the noun in the first transcription or in the response to the first utterance; and

based on determining whether the second utterance includes a query directed to the query processing system, configuring, by the computing device, a data routing component to (i) provide the second transcription of the second utterance to the query processing system as a second query or (ii) bypass routing the second transcription so that the second transcription is not provided to the query processing system.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2019
From: HOWARD, NATHAN DAVID; SIMKO, GABOR; GIURGIU, ANDREI; BEHZADI, BEHSHAD; NOWAK-PRZYGODZKI, MARCIN M.
To: GOOGLE LLC
Reel/Frame 051162/0434 →
Continuity (2)
Continuation PCTUS2019019829 · Feb 27, 2019
Related Publication 20200272690A1 · Aug 27, 2020
Cited By (1)
US 12,223,950