Background audio identification for speech disambiguation
Implementations relate to techniques for providing context-dependent search results. A computer-implemented method includes receiving an audio stream at a computing device during a time interval, the audio stream comprising user speech data and background audio, separating the audio stream into a first substream that includes the user speech data and a second substream that includes the background audio, identifying concepts related to the background audio, generating a set of terms related to the identified concepts, influencing a speech recognizer based on at least one of the terms related to the background audio, and obtaining a recognized version of the user speech data using the speech recognizer.
1. A computer-implemented method comprising:
receiving an audio stream containing first audio data and second audio data, the first audio data corresponding to an utterance spoken by a user and the second audio data corresponding to background audio associated with playback of an item of media content;
processing the second audio data to generate at least one term associated with the background audio;
adjusting a probability or relevance score associated with a speech recognition model recognizing the at least one term associated with the background audio in the first audio data;
after adjusting the probability or relevance score associated with the speech recognition model recognizing the at least one term associated with the background audio, transcribing the first audio data into a textual representation of the utterance using the speech recognition model; and
transmitting the textual representation of the utterance to a computing device associated with the user, the textual representation of the utterance when received by the computing device causing the computing device to at least one of display the textual representation on a display or perform a particular task based on the textual representation.
2. The computer-implemented method of claim 1 , wherein processing the second audio data to generate the at least one term comprises:
identifying one or more concepts from audio features of the second audio data; and
generating a set of terms related to the identified one or more concepts.
3. The computer-implemented method of claim 2 , wherein generating the set of terms related to the identified one or more concepts comprises querying a conceptual expansion database for the set of terms using the one or more concepts identified from the audio features of the second audio data.
4. The computer-implemented method of claim 1 , wherein receiving the audio stream containing the first audio data and the second audio data comprises receiving the first audio data and the second audio data from the computing device associated with the user, the first audio data and the second audio data captured by the computing device during a same time interval.
5. The computer-implemented method of claim 2 , further comprising:
generating conceptual bias data using the at least one term associated with the background audio; and
adjusting the probability or relevance score associated with the speech recognition model recognizing the at least one term associated with the background audio in the first audio data based on the conceptual bias data.
6. The computer-implemented method of claim 2 , wherein identifying the one or more concepts from the audio features of the second audio data comprises:
comparing the audio features of the second audio data to known audio segments;
determining that the audio features of the second audio data correspond to audio features of a particular known audio segment; and
identifying the one or more concepts as being related to the particular known audio segment.
7. The computer-implemented method of claim 5 , wherein transcribing the first audio data into the textual representation of the utterance comprises selecting, by the speech recognition model, the textual representation from a set of textual representations that have substantially similar frequencies of occurrence in a particular language by using the conceptual data to weigh a statistical selection of the textual representation from the set of textual representations.
8. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving an audio stream containing first audio data and second audio data, the first audio data corresponding to an utterance spoken by a user and the second audio data corresponding to background audio associated with playback of an item of media content;
processing the second audio data to generate at least one term associated with the background audio;
adjusting a probability or relevance score associated with a speech recognition model recognizing the at least one term associated with the background audio in the first audio data;
after adjusting the probability or relevance score associated with the speech recognition model recognizing the at least one term associated with the background audio, transcribing the first audio data into a textual representation of the utterance using the speech recognition model; and
transmitting the textual representation of the utterance to a computing device associated with the user, the textual representation of the utterance when received by the computing device causing the computing device to at least one of display the textual representation on a display or perform a particular task based on the textual representation.
9. The system of claim 8 , wherein processing the second audio data to generate the at least one term comprises:
identifying one or more concepts from audio features of the second audio data; and
generating a set of terms related to the identified one or more concepts.
10. The system of claim 9 , wherein generating the set of terms related to the identified one or more concepts comprises querying a conceptual expansion database for the set of terms using the one or more concepts identified from the audio features of the second audio data.
11. The system of claim 8 , wherein receiving the audio stream containing the first audio data and the comprises receiving the first audio data and the second audio data from the computing device associated with the user, the first audio data and the second audio data captured by the computing device during a same time interval.
12. The system of claim 9 , wherein the operations further comprise:
generating conceptual bias data using the at least one term associated with the background audio; and
adjusting the probability or relevance score associated with the speech recognition model recognizing the at least one term associated with the background audio in the first audio data based on the conceptual bias data.
13. The system of claim 9 , wherein identifying the one or more concepts from the audio features of the comprises:
comparing the audio features of the second audio data to known audio segments;
determining that the audio features of the second audio data correspond to audio features of a particular known audio segment; and
identifying the one or more concepts as being related to the particular known audio segment.
14. The system of claim 12 , wherein transcribing the first audio data into the textual representation of the utterance comprises selecting, by the speech recognition model, the textual representation from a set of textual representations that have substantially similar frequencies of occurrence in a particular language by using the conceptual data to weigh a statistical selection of the textual representation from the set of textual representations.
15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
receiving an audio stream containing first audio data and second audio data, the first audio data corresponding to an utterance spoken by a user and the second audio data corresponding to background audio associated with playback of an item of media content;
processing the second audio data to generate at least one term associated with the background audio;
adjusting a probability or relevance score associated with a speech recognition model recognizing the at least one term associated with the background audio in the first audio data;
after adjusting the probability or relevance score associated with the speech recognition model recognizing the at least one term associated with the background audio, transcribing the first audio data into a textual representation of the utterance using the speech recognition model; and
transmitting the textual representation of the utterance to a computing device associated with the user, the textual representation of the utterance when received by the computing device causing the computing device to at least one of display the textual representation on a display or perform a particular task based on the textual representation.
16. The computer-readable medium of claim 15 , wherein processing the second audio data to generate the at least one term comprises:
identifying one or more concepts from audio features of the second audio data; and
generating a set of terms related to the identified one or more concepts.
17. The computer-readable medium of claim 16 , wherein generating the set of terms related to the identified one or more concepts comprises querying a conceptual expansion database for the set of terms using the one or more concepts identified from the audio features of the second audio data.
18. The computer-readable medium of claim 15 , wherein receiving the audio stream containing the first audio data and the comprises receiving the first audio data and the second audio data from the computing device associated with the user, the first audio data and the second audio data captured by the computing device during a same time interval.
19. The computer-readable medium of claim 16 , wherein the operations further comprise:
generating conceptual bias data using the at least one term associated with the background audio; and
adjusting the probability or relevance score associated with the speech recognition model recognizing the at least one term associated with the background audio in the first audio data based on the conceptual bias data.
20. The computer-readable medium of claim 16 , wherein identifying the one or more concepts from the audio features of the comprises:
comparing the audio features of the second audio data to known audio segments;
determining that the audio features of the second audio data correspond to audio features of a particular known audio segment; and
identifying the one or more concepts as being related to the particular known audio segment.