IP Library › Granted Patent US 10,546,576
Granted Patent B2
US 10,546,576 · App. 16/154,875 · Granted Jan 28, 2020

Speech endpointing based on word comparisons

Inventors: Michael Buchanan (Palo Alto, CA); Pravir Kumar Gupta (Los Altos, CA); Christopher Bo Tandiono (Fremont, CA)
Assignee: Google LLC
G10L15/05G10L15/04G10L15/22G10L15/26G10L17/06G10L25/51G10L25/87G10L25/78G10L25/90G10L2015/088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,546,576
App. No.
16/154,875
Granted
Jan 28, 2020
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for speech endpointing based on word comparisons are described. In one aspect, a method includes the actions of obtaining a transcription of an utterance. The actions further include determining, as a first value, a quantity of text samples in a collection of text samples that (i) include terms that match the transcription, and (ii) do not include any additional terms. The actions further include determining, as a second value, a quantity of text samples in the collection of text samples that (i) include terms that match the transcription, and (ii) include one or more additional terms. The actions further include classifying the utterance as a likely incomplete utterance or not a likely incomplete utterance based at least on comparing the first value and the second value.

Claims (71)

1. A computer-implemented method comprising:

receiving, by a computing device, audio that includes an utterance spoken by a user

determining, by the computing device, that an energy level of the audio that includes the utterance is above a threshold energy level;

while the computing device receives the audio that includes the utterance and while the energy level of the audio that includes the utterance remains above the threshold energy level, determining, by the computing device, to delay designating an endpoint of the utterance spoken by the user until the energy level of the audio that includes the utterance is below the threshold energy level;

while the computing device receives the audio that includes the utterance and while the energy level of the audio that includes the utterance remains above the threshold energy level, obtaining, by the computing device, a transcription of the utterance spoken by the user; and

while the computing device receives the audio that includes the utterance, while the energy level of the audio that includes the utterance remains above the threshold energy level, and based on the transcription of the utterance, overriding, by the computing device, the determination to delay designating an endpoint of the utterance spoken by the user and designating an endpoint of the utterance spoken by the user.

2. The method of claim 1 , comprising:

determining, by the computing device, that the utterance is likely complete,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on determining that the utterance is likely complete.

3. The method of claim 1 , comprising:

comparing, by the computing device, the transcription to a collection of text samples,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on comparing the transcription to a collection of text samples.

4. The method of claim 3 , comprising:

based on comparing the transcription to a collection of text samples determining a number of text samples in the collection of text samples that match the transcription and do not include any additional terms,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on the number of text samples in the collection of text samples that match the transcription and do not include any additional terms.

5. The method of claim 4 , comprising:

determining that terms in each of the text samples in the collection of text samples that match the transcription and do not include any additional terms occur in a same order as in the transcription,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on determining that the terms in each of the text samples in the collection of text samples that match the transcription and do not include any additional terms occur in the same order as in the transcription.

6. The method of claim 3 , comprising:

based on comparing the transcription to a collection of text samples determining a number of text samples in the collection of text samples that match the transcription and include one or more additional terms,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on the number of text samples in the collection of text samples that match the transcription and include one or more additional terms.

7. The method of claim 1 , wherein:

receiving the audio that includes the utterance spoken by the user and that has the energy level that is above the threshold energy level comprises receiving, by a microphone of the computing device, the audio that includes the utterance, and

overriding the determination to delay designating an endpoint of the utterance spoken by the user and designating an endpoint of the utterance comprises deactivating, by the computing device, the microphone while the energy level of the audio that includes the utterance and that is received by the microphone remains above the threshold energy level.

8. A system comprising:

one or more computers; and

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving, by a computing device, audio that includes an utterance spoken by a user

determining, by the computing device, that an energy level of the audio that includes the utterance is above a threshold energy level;

while the computing device receives the audio that includes the utterance and while the energy level of the audio that includes the utterance remains above the threshold energy level, determining, by the computing device, to delay designating an endpoint of the utterance spoken by the user until the energy level of the audio that includes the utterance is below the threshold energy level;

while the computing device receives the audio that includes the utterance and while the energy level of the audio that includes the utterance remains above the threshold energy level, obtaining, by the computing device, a transcription of the utterance spoken by the user; and

while the computing device receives the audio that includes the utterance, while the energy level of the audio that includes the utterance remains above the threshold energy level, and based on the transcription of the utterance, overriding, by the computing device, the determination to delay designating an endpoint of the utterance spoken by the user and designating an endpoint of the utterance spoken by the user.

9. The system of claim 8 , wherein the operations comprise:

determining, by the computing device, that the utterance is likely complete,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on determining that the utterance is likely complete.

10. The system of claim 8 , wherein the operations comprise:

comparing, by the computing device, the transcription to a collection of text samples,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on comparing the transcription to a collection of text samples.

11. The system of claim 10 , wherein the operations comprise:

based on comparing the transcription to a collection of text samples determining a number of text samples in the collection of text samples that match the transcription and do not include any additional terms,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on the number of text samples in the collection of text samples that match the transcription and do not include any additional terms.

12. The system of claim 11 , wherein the operations comprise:

determining that terms in each of the text samples in the collection of text samples that match the transcription and do not include any additional terms occur in a same order as in the transcription,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on determining that the terms in each of the text samples in the collection of text samples that match the transcription and do not include any additional terms occur in the same order as in the transcription.

13. The system of claim 10 , wherein the operations comprise:

based on comparing the transcription to a collection of text samples determining a number of text samples in the collection of text samples that match the transcription and include one or more additional terms,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on the number of text samples in the collection of text samples that match the transcription and include one or more additional terms.

14. The system of claim 8 , wherein:

receiving the audio that includes the utterance spoken by the user and that has the energy level that is above the threshold energy level comprises receiving, by a microphone of the computing device, the audio that includes the utterance, and

overriding the determination to delay designating an endpoint of the utterance spoken by the user and designating an endpoint of the utterance comprises deactivating, by the computing device, the microphone while the energy level of the audio that includes the utterance and that is received by the microphone remains above the threshold energy level.

15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a computing device, audio that includes an utterance spoken by a user

determining, by the computing device, that an energy level of the audio that includes the utterance is above a threshold energy level;

while the computing device receives the audio that includes the utterance and while the energy level of the audio that includes the utterance remains above the threshold energy level, determining, by the computing device, to delay designating an endpoint of the utterance spoken by the user until the energy level of the audio that includes the utterance is below the threshold energy level;

while the computing device receives the audio that includes the utterance and while the energy level of the audio that includes the utterance remains above the threshold energy level, obtaining, by the computing device, a transcription of the utterance spoken by the user; and

while the computing device receives the audio that includes the utterance, while the energy level of the audio that includes the utterance remains above the threshold energy level, and based on the transcription of the utterance, overriding, by the computing device, the determination to delay designating an endpoint of the utterance spoken by the user and designating an endpoint of the utterance spoken by the user.

16. The medium of claim 15 , wherein the operations comprise:

determining, by the computing device, that the utterance is likely complete,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on determining that the utterance is likely complete.

17. The medium of claim 15 , wherein the operations comprise:

comparing, by the computing device, the transcription to a collection of text samples,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on comparing the transcription to a collection of text samples.

18. The medium of claim 17 , wherein the operations comprise:

based on comparing the transcription to a collection of text samples determining a number of text samples in the collection of text samples that match the transcription and do not include any additional terms,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on the number of text samples in the collection of text samples that match the transcription and do not include any additional terms.

19. The medium of claim 17 , wherein the operations comprise:

based on comparing the transcription to a collection of text samples determining a number of text samples in the collection of text samples that match the transcription and include one or more additional terms,

wherein the computing device overrides the determination to delay designating an endpoint of the utterance spoken by the user and designates an endpoint of the utterance based on the number of text samples in the collection of text samples that match the transcription and include one or more additional terms.

20. The medium of claim 15 , wherein:

receiving the audio that includes the utterance spoken by the user and that has the energy level that is above the threshold energy level comprises receiving, by a microphone of the computing device, the audio that includes the utterance, and

overriding the determination to delay designating an endpoint of the utterance spoken by the user and designating an endpoint of the utterance comprises deactivating, by the computing device, the microphone while the energy level of the audio that includes the utterance and that is received by the microphone remains above the threshold energy level.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2018
From: BUCHANAN, MICHAEL; GUPTA, PRAVIR KUMAR; TANDIONO, CHRISTOPHER BO
To: GOOGLE INC.
Reel/Frame 047108/0742 →
ENTITY CONVERSION Recorded Oct 9, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 047207/0950 →
Continuity (4)
Continuation 15156478 · May 17, 2016
Continuation 14681203 · Apr 8, 2015
Provisional Application 61983025 · Apr 23, 2014
Related Publication 20190043480A1 · Feb 7, 2019
Cited By (2)
US 12,536,996 US 12,562,156