Recognizing speech in the presence of additional audio
The technology described in this document can be embodied in a computer-implemented method that includes receiving, at a processing system, a first signal including an output of a speaker device and an additional audio signal. The method also includes determining, by the processing system, based at least in part on a model trained to identify the output of the speaker device, that the additional audio signal corresponds to an utterance of a user. The method further includes initiating a reduction in an audio output level of the speaker device based on determining that the additional audio signal corresponds to the utterance of the user.
1. A computer-implemented method comprising:
receiving, by a computing device that includes (i) a text-to-speech engine, (ii) an automated speech recognizer, and (iii) a barge-in detection model that is trained to output an indication of whether given audio data comprises synthesized speech, an audio signal corresponding to a user's utterance that is spoken while the computing device is outputting synthesized speech;
processing, using the barge-in detection model that is trained to output an indication of whether given audio data comprises synthesized speech, particular audio data that comprises data corresponding to the audio signal and data corresponding to the synthesized speech;
in response to receiving an indication from the barge-in model that the particular audio data comprises synthesized speech, suppressing a further output of the text-to-speech engine; and
outputting, by the automated speech recognizer, a transcription of the user's utterance without the synthesized speech.
2. The method of claim 1 , wherein processing the particular audio data comprises:
determining an obtained vector corresponding to the audio signal received by the computing device;
comparing the obtained vector corresponding to the audio signal to a reference vector corresponding to the barge-in model; and
determining that the audio signal likely includes synthesized speech based on a result of comparing the obtained vector to the reference vector satisfying a predetermined threshold.
3. The method of claim 2 , wherein comparing the obtained vector corresponding to the audio signal to a reference vector corresponding to a model that is trained to identify whether given audio data comprises additional audio other than a synthesized voice comprises calculating a cosine distance between the obtained vector and the reference vector, and
wherein the predetermined threshold corresponds to a value representing a threshold cosine distance between the obtained vector and the reference vector.
4. The method of claim 1 , wherein suppressing the further output of the text-to-speech engine comprises initiating a reduction in an audio output level of the text-to-speech engine.
5. The method of claim 1 , wherein suppressing the further output of the text-to-speech engine comprises at least temporarily ceasing an audio output of the text-to-speech engine.
6. The method of claim 1 , wherein the barge-in model is trained to identify whether given audio data comprises a user's voice.
7. The method of claim 1 , wherein the data corresponding to the audio signal comprises an i-vector.
8. A system comprising:
one or more computers; and
one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving, by a computing device that includes (i) a text-to-speech engine, (ii) an automated speech recognizer, and (iii) a barge-in detection model that is trained to output an indication of whether given audio data comprises synthesized speech, an audio signal corresponding to a user's utterance that is spoken while the computing device is outputting synthesized speech;
processing, using the barge-in detection model that is trained to output an indication of whether given audio data comprises synthesized speech, particular audio data that comprises data corresponding to the audio signal and data corresponding to the synthesized speech;
in response to receiving an indication from the barge-in model that the particular audio data comprises synthesized speech, suppressing a further output of the text-to-speech engine; and
outputting, by the automated speech recognizer, a transcription of the user's utterance without the synthesized speech.
9. The system of claim 8 , wherein processing the particular audio data comprises:
determining an obtained vector corresponding to the audio signal received by the computing device;
comparing the obtained vector corresponding to the audio signal to a reference vector corresponding to the barge-in model; and
determining that the audio signal likely includes synthesized speech based on a result of comparing the obtained vector to the reference vector satisfying a predetermined threshold.
10. The system of claim 9 , wherein comparing the obtained vector corresponding to the audio signal to a reference vector corresponding to a model that is trained to identify whether given audio data comprises additional audio other than a synthesized voice comprises calculating a cosine distance between the obtained vector and the reference vector, and
wherein the predetermined threshold corresponds to a value representing a threshold cosine distance between the obtained vector and the reference vector.
11. The system of claim 8 , wherein suppressing the further output of the text-to-speech engine comprises initiating a reduction in an audio output level of the text-to-speech engine.
12. The system of claim 8 , wherein suppressing the further output of the text-to-speech engine comprises at least temporarily ceasing an audio output of the text-to-speech engine.
13. The system of claim 8 , wherein the barge-in model is trained to identify whether given audio data comprises a user's voice.
14. The system of claim 8 , wherein the data corresponding to the audio signal comprises an i-vector.
15. A non-transitory computer readable storage device storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving, by a computing device that includes (i) a text-to-speech engine, (ii) an automated speech recognizer, and (iii) a barge-in detection model that is trained to output an indication of whether given audio data comprises synthesized speech, an audio signal corresponding to a user's utterance that is spoken while the computing device is outputting synthesized speech;
processing, using the barge-in detection model that is trained to output an indication of whether given audio data comprises synthesized speech, particular audio data that comprises data corresponding to the audio signal and data corresponding to the synthesized speech;
in response to receiving an indication from the barge-in model that the particular audio data comprises synthesized speech, suppressing a further output of the text-to-speech engine; and
outputting, by the automated speech recognizer, a transcription of the user's utterance without the synthesized speech.
16. The device of claim 15 , wherein processing the particular audio data comprises:
determining an obtained vector corresponding to the audio signal received by the computing device;
comparing the obtained vector corresponding to the audio signal to a reference vector corresponding to the barge-in model; and
determining that the audio signal likely includes synthesized speech based on a result of comparing the obtained vector to the reference vector satisfying a predetermined threshold.
17. The device of claim 16 , wherein comparing the obtained vector corresponding to the audio signal to a reference vector corresponding to a model that is trained to identify whether given audio data comprises additional audio other than a synthesized voice comprises calculating a cosine distance between the obtained vector and the reference vector, and
wherein the predetermined threshold corresponds to a value representing a threshold cosine distance between the obtained vector and the reference vector.
18. The device of claim 15 , wherein suppressing the further output of the text-to-speech engine comprises initiating a reduction in an audio output level of the text-to-speech engine.
19. The device of claim 15 , wherein suppressing the further output of the text-to-speech engine comprises at least temporarily ceasing an audio output of the text-to-speech engine.
20. The device of claim 15 , wherein the barge-in model is trained to identify whether given audio data comprises a user's voice.