Recognizing speech in the presence of additional audio
The technology described in this document can be embodied in a computer-implemented method that includes receiving, at a processing system, a first signal including an output of a speaker device and an additional audio signal. The method also includes determining, by the processing system, based at least in part on a model trained to identify the output of the speaker device, that the additional audio signal corresponds to an utterance of a user. The method further includes initiating a reduction in an audio output level of the speaker device based on determining that the additional audio signal corresponds to the utterance of the user.
1. A computer-implemented method comprising:
receiving an audio signal;
determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice;
in response to determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing operation of a speech synthesis module;
after suppressing operation of the speech synthesis module, obtaining a transcription corresponding to the audio signal from an automated speech recognizer; and
providing the transcription for output.
2. The method of claim 1 , wherein suppressing operation of a speech synthesis module comprises initiating a reduction in an audio output level of the speech synthesis module.
3. The method of claim 2 , wherein initiating a reduction in an audio output level of the speech synthesis module comprises interrupting output of the speech synthesis module.
4. The method of claim 1 , comprising:
obtaining a first vector corresponding to at least a portion of the audio signal;
comparing the first vector to a second vector corresponding to the model that is trained to detect a presence of a synthesized voice; and
determining that the audio signal comprises additional audio other than the synthesized voice based on a result of the comparison satisfying a threshold.
5. The method of claim 1 , comprising:
obtaining a first vector corresponding to at least a portion of the audio signal; and
determining that the audio signal comprises additional audio other than the synthesized voice based on the first vector satisfying a threshold.
6. The method of claim 1 , wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user's voice is an i-vector based model.
7. The method of claim 1 , wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user's voice is a neural network based model.
8. The method of claim 1 , wherein:
receiving an audio signal comprises receiving, by an assistant device, an audio signal,
determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice comprises determining, by the assistant device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice,
in response to determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing operation of a speech synthesis module comprises, in response to determining, by the assistant device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing, by the assistant device, operation of a speech synthesis module implemented by the assistant device,
after suppressing operation of the speech synthesis module, obtaining a transcription corresponding to the audio signal from an automated speech recognizer comprises, after suppressing operation of the speech synthesis module, obtaining, by the assistant device, a transcription corresponding to the audio signal from an automated speech recognizer, and
providing the transcription for output comprises providing, by the assistant device, the transcription for output.
9. A system comprising:
one or more computers; and
one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving an audio signal;
determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice;
in response to determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing operation of a speech synthesis module;
after suppressing operation of the speech synthesis module, obtaining a transcription corresponding to the audio signal from an automated speech recognizer; and
providing the transcription for output.
10. The system of claim 9 , wherein suppressing operation of a speech synthesis module comprises initiating a reduction in an audio output level of the speech synthesis module.
11. The system of claim 10 , wherein initiating a reduction in an audio output level of the speech synthesis module comprises interrupting output of the speech synthesis module.
12. The system of claim 9 , wherein the operations further comprise:
obtaining a first vector corresponding to at least a portion of the audio signal;
comparing the first vector to a second vector corresponding to the model that is trained to detect a presence of a synthesized voice; and
determining that the audio signal comprises additional audio other than the synthesized voice based on a result of the comparison satisfying a threshold.
13. The system of claim 9 , wherein the operations further comprise:
obtaining a first vector corresponding to at least a portion of the audio signal; and
determining that the audio signal comprises additional audio other than the synthesized voice based on the first vector satisfying a threshold.
14. The system of claim 9 , wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user's voice is an i-vector based model.
15. The system of claim 9 , wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user's voice is a neural network based model.
16. The system of claim 9 , wherein:
receiving an audio signal comprises receiving, by an assistant device, an audio signal,
determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice comprises determining, by the assistant device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice,
in response to determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing operation of a speech synthesis module comprises, in response to determining, by the assistant device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing, by the assistant device, operation of a speech synthesis module implemented by the assistant device,
after suppressing operation of the speech synthesis module, obtaining a transcription corresponding to the audio signal from an automated speech recognizer comprises, after suppressing operation of the speech synthesis module, obtaining, by the assistant device, a transcription corresponding to the audio signal from an automated speech recognizer, and
providing the transcription for output comprises providing, by the assistant device, the transcription for output.
17. A non-transitory computer readable storage device storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving an audio signal;
determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice;
in response to determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing operation of a speech synthesis module;
after suppressing operation of the speech synthesis module, obtaining a transcription corresponding to the audio signal from an automated speech recognizer; and
providing the transcription for output.
18. The device of claim 17 , wherein the operations further comprise:
obtaining a first vector corresponding to at least a portion of the audio signal;
comparing the first vector to a second vector corresponding to the model that is trained to detect a presence of a synthesized voice; and
determining that the audio signal comprises additional audio other than the synthesized voice based on a result of the comparison satisfying a threshold.
19. The device of claim 17 , wherein the operations further comprise:
obtaining a first vector corresponding to at least a portion of the audio signal; and
determining that the audio signal comprises additional audio other than the synthesized voice based on the first vector satisfying a threshold.
20. The device of claim 17 , wherein:
receiving an audio signal comprises receiving, by an assistant device, an audio signal,
determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice comprises determining, by the assistant device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice,
in response to determining, using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing operation of a speech synthesis module comprises, in response to determining, by the assistant device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user's voice, that the audio signal likely includes both the synthesized voice and the user's voice, suppressing, by the assistant device, operation of a speech synthesis module implemented by the assistant device,
after suppressing operation of the speech synthesis module, obtaining a transcription corresponding to the audio signal from an automated speech recognizer comprises, after suppressing operation of the speech synthesis module, obtaining, by the assistant device, a transcription corresponding to the audio signal from an automated speech recognizer, and
providing the transcription for output comprises providing, by the assistant device, the transcription for output.