Recognizing speech in the presence of additional audio
The technology described in this document can be embodied in a computer-implemented method that includes receiving, at a processing system, a first signal including an output of a speaker device and an additional audio signal. The method also includes determining, by the processing system, based at least in part on a model trained to identify the output of the speaker device, that the additional audio signal corresponds to an utterance of a user. The method further includes initiating a reduction in an audio output level of the speaker device based on determining that the additional audio signal corresponds to the utterance of the user.
1. A computer-implemented method executed on data processing hardware that causes the data processing hardware to perform operations comprising:
receiving a first query spoken by a user and captured by a microphone of a computing device associated with the user;
providing, for audible playback from the computing device, a text-to-speech (TTS) output generated by a TTS system associated with the computing device, the TTS output comprising synthesized audio that conveys a response to the first query;
while the computing device is audibly playing back the TTS output:
detecting a barge-in event from the user to provide a second query;
in response to detecting the barge-in event, initiating a reduction in an audio output level of the computing device; and
receiving an audio signal captured by the microphone that conveys the second query spoken by the user; and
providing the audio signal characterizing the second query to a speech recognition engine.
2. The computer-implemented method of claim 1 , wherein a language model is configured to process an output of the speech recognition engine.
3. The computer-implemented method of claim 2 , wherein the output of the speech recognition engine comprises a vector of ordered pairs of speech units in a given language and corresponding weights.
4. The computer-implemented method of claim 3 , wherein the language model is configured to process the output of the speech recognition engine by determining a likelihood of word and phrase sequences from the speech units in the given language and the corresponding weights.
5. The computer-implemented method of claim 1 , wherein initiating the reduction in the audio output level of the computing device comprises providing a control signal to the computing device that causes the computing device to switch-off audible playback of the TTS output.
6. The computer-implemented method of claim 1 , wherein the data processing hardware is implemented on the computing device.
7. The computer-implemented method of claim 1 , wherein the speech recognition engine is configured to generate a transcription of the second query spoken by the user.
8. The computer-implemented method of claim 7 , wherein the operations further comprise:
transforming the transcription of the second query into a structured representation; and
processing, using a particular application, the structured representation.
9. The computer-implemented method of claim 1 , wherein the computing device comprises a mobile phone.
10. The computer-implemented method of claim 1 , wherein the computing device comprises a speaker device.
11. A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware causes the data processing hardware to perform operations comprising:
receiving a first query spoken by a user and captured by a microphone of a computing device associated with the user;
providing, for audible playback from the computing device, a text-to-speech (TTS) output generated by a TTS system associated with the computing device, the TTS output comprising synthesized audio that conveys a response to the first query;
while the computing device is audibly playing back the TTS output:
detecting a barge-in event from the user to provide a second query;
in response to detecting the barge-in event, initiating a reduction in an audio output level of the computing device; and
receiving an audio signal captured by the microphone that conveys the second query spoken by the user; and
providing the audio signal characterizing the second query to a speech recognition engine.
12. The system of claim 11 , wherein a language model is configured to process an output of the speech recognition engine.
13. The system of claim 12 , wherein the output of the speech recognition engine comprises a vector of ordered pairs of speech units in a given language and corresponding weights.
14. The system of claim 13 , wherein the language model is configured to process the output of the speech recognition engine by determining a likelihood of word and phrase sequences from the speech units in the given language and the corresponding weights.
15. The system of claim 11 , wherein initiating the reduction in the audio output level of the computing device comprises providing a control signal to the computing device that causes the computing device to switch-off audible playback of the TTS output.
16. The system of claim 11 , wherein the data processing hardware is implemented on the computing device.
17. The system of claim 11 , wherein the speech recognition engine is configured to generate a transcription of the second query spoken by the user.
18. The system of claim 17 , wherein the operations further comprise:
transforming the transcription of the second query into a structured representation; and
processing, using a particular application, the structured representation.
19. The system of claim 11 , wherein the computing device comprises a mobile phone.
20. The system of claim 11 , wherein the computing device comprises a speaker device.