Speech endpointing
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for speech endpointing are described. In one aspect, a method includes the action of accessing voice query log data that includes voice queries spoken by a particular user. The actions further include based on the voice query log data that includes voice queries spoken by a particular user, determining a pause threshold from the voice query log data that includes voice queries spoken by the particular user. The actions further include receiving, from the particular user, an utterance. The actions further include determining that the particular user has stopped speaking for at least a period of time equal to the pause threshold. The actions further include based on determining that the particular user has stopped speaking for at least a period of time equal to the pause threshold, processing the utterance as a voice query.
1. A computer-implemented method comprising:
accessing, by one or more computing devices, a collection of voice queries that were submitted by a user;
determining, by the one or more computing devices, a pause duration threshold for the particular user based on durations of pauses between words of the voice queries in the collection of voice queries;
receiving, by the one or more computing devices, through a microphone of a computing device associated with the user, audio data corresponding to an utterance spoken by the particular user; and
processing the audio data by:
transcribing, by a speech recognizer, each word in the utterance as the audio data is received;
detecting a pause in the audio data indicating when the user is not speaking;
determining whether a duration of the pause detected in the audio data satisfies the pause duration threshold;
in response to determining that the duration of the pause detected in the audio data satisfies the pause duration threshold, determining whether a sequence of one or more words transcribed by the speech recognizer represents a complete utterance previously spoken by the particular user or another user; and
when the sequence of one or more words transcribed by the speech recognizer represents the complete utterance:
triggering an endpointer to endpoint the audio data by designating a temporal location in the audio data; and
processing, using a natural language processing system, the endpointed audio data as a voice query, the endpointed audio data including audio data before the temporal location in the audio data and excluding audio data after the temporal location in the audio data.
2. The method of claim 1 , further comprising designating the pause duration threshold as a user-specific endpointing threshold for the particular user.
3. The method of claim 1 , further comprising, in response to determining that the duration of the pause detected in the audio data satisfies the pause duration threshold and when the sequence of one or more words transcribed by the speech recognizer does not represent the complete utterance, continuing processing the audio data without endpointing the audio data.
4. The method of claim 1 , wherein the pause duration threshold is less than a default pause duration threshold designated for one or more other users.
5. The method of claim 4 , wherein the duration of the pause detected in the audio data is greater than the pause duration threshold and less than the default pause duration threshold.
6. The method of claim 1 , wherein the pause duration threshold is greater than a default pause duration threshold designated for one or more other users.
7. The method of claim 6 , wherein the duration of the pause detected in the audio data is greater than the default pause duration threshold and less than the pause duration threshold.
8. The method of claim 1 , further comprising:
determining an average length of the voice queries in the collection of voice queries; and
modifying the pause duration threshold based on the average length.
9. The method of claim 1 , further comprising:
determining an average number of the voice queries in the collection of voice queries that were spoken each day by the user; and
modifying the pause duration threshold based on the average number of the voice queries in the collection of voice queries that were spoken each day by the user.
10. The method of claim 1 , further comprising:
classifying the voice query in a collection of complete voice queries; and
modifying the pause duration threshold based on the classification.
11. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
accessing a collection of voice queries that were submitted by a user;
determining a pause duration threshold for the particular user based on durations of pauses between words of the voice queries in the collection of voice queries;
receiving, through a microphone of a computing device associated with the user, audio data corresponding to an utterance spoken by the particular user;
processing the audio data by:
transcribing, by a speech recognizer, each word in the utterance as the audio data is received;
detecting a pause in the audio data indicating when the user is not speaking;
determining whether a duration of the pause detected in the audio data satisfies the pause duration threshold;
in response to determining that the duration of the pause detected in the audio data satisfies the pause duration threshold, determining whether a sequence of one or more words transcribed by the speech recognizer represents a complete utterance previously spoken by the particular user or another user; and
when the sequence of one or more words transcribed by the speech recognizer represents the complete utterance:
triggering an endpointer to endpoint the audio data by designating a temporal location in the audio data; and
processing, using a natural language processing system, the endpointed audio data as a voice query, the endpointed audio data including audio data before the temporal location in the audio data and excluding audio data after the temporal location in the audio data.
12. The system of claim 11 , wherein the operations further comprise designating the pause duration threshold as a user-specific endpointing threshold for the particular user.
13. The system of claim 11 , further comprising, in response to determining that the duration of the pause detected in the audio data satisfies the pause duration threshold and when the sequence of one or more words transcribed by the speech recognizer does not represent the complete utterance, continuing processing the audio data without endpointing the audio data.
14. The system of claim 11 , wherein the pause duration threshold is less than a default pause duration threshold designated for one or more other users.
15. The system of claim 14 , wherein the duration of the pause detected in the audio data is greater than the pause duration threshold and less than the default pause duration threshold.
16. The system of claim 11 , wherein the duration value is greater than a default pause duration threshold designated for one or more other users.
17. The system of claim 16 , wherein the duration of the pause detected in the audio data is greater than the default pause duration threshold and less than the pause duration threshold.
18. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
accessing a collection of voice queries that were submitted by a user;
determining a pause duration threshold for the particular user based on durations of pauses between words of the voice queries in the collection of voice queries;
receiving, through a microphone of a computing device associated with the user, audio data corresponding to an utterance spoken by the particular user; and
processing the audio data by:
transcribing, by a speech recognizer, each word in the utterance as the audio data is received;
detecting a pause in the audio data indicating when the user is not speaking;
determining whether a duration of the pause detected in the audio data satisfies the pause duration threshold;
in response to determining that the duration of the pause detected in the audio data satisfies the pause duration threshold, determining whether a sequence of one or more words transcribed by the speech recognizer represents a complete utterance previously spoken by the particular user or another user; and
when the sequence of one or more words transcribed by the speech recognizer represents the complete utterance:
triggering an endpointer to endpoint the audio data by designating a temporal location in the audio data; and
processing, using a natural language processing system, the endpointed audio data as a voice query, the endpointed audio data including audio data before the temporal location in the audio data and excluding audio data after the temporal location in the audio data.