Determining dialog states for language models
Systems, methods, devices, and other techniques are described herein for determining dialog states that correspond to voice inputs and for biasing a language model based on the determined dialog states. In some implementations, a method includes receiving, at a computing system, audio data that indicates a voice input and determining a particular dialog state, from among a plurality of dialog states, which corresponds to the voice input. A set of n-grams can be identified that are associated with the particular dialog state that corresponds to the voice input. In response to identifying the set of n-grams that are associated with the particular dialog state that corresponds to the voice input, a language model can be biased by adjusting probability scores that the language model indicates for n-grams in the set of n-grams. The voice input can be transcribed using the adjusted language model.
1. A computer-implemented method that when executed on data processing hardware causes the data processing hardware to perform operations comprising:
receiving a transcription request for a voice input captured by a user device, the transcription request comprising audio data and context data, the audio data indicating the voice input and the context data indicating:
an application to which the voice input is directed; and
a particular stage from a multi-stage voice activity corresponding to a series of user interactions related to a task of the application;
based on the context data, determining, from among multiple possible dialogs, a particular dialog corresponding to the particular stage from the multi-stage voice activity;
based on the particular dialog corresponding to the particular stage from the multi-stage voice activity, filtering the audio data to only include audio data associated with the particular dialog;
identifying, from a set of n-grams, a representative subset of n-grams associated with the particular dialog corresponding to the particular stage from the multi-stage voice activity based on one or more prior stages from the multi-stage voice activity corresponding to one or more prior transcription requests, each n-gram of the set of n-grams comprising a respective non-zero probability score;
biasing a language model by increasing the respective non-zero probability score for each n-gram of the identified representative subset of n-grams associated with the particular dialog; and
processing, using the biased language model, the filtered audio data to generate a transcription of the voice input.
2. The method of claim 1 , wherein the identified representative subset of n-grams associated with the particular dialog corresponding to the particular stage from the multi-stage voice activity is further based at least on n-grams in the representative subset of n-grams occurring frequently in historical voice inputs that correspond to the particular dialog.
3. The method of claim 1 , wherein the operations further comprise:
generating an initial transcription of the voice input using the language model prior to biasing the language model,
wherein determining the particular dialog corresponding to the particular stage from the multi-stage voice activity is further based on the initial transcription.
4. The method of claim 1 , wherein the operations further comprise:
receiving display data associated with a user interface of the user device at a time the voice input was captured by the user device,
wherein determining the particular dialog corresponding to the particular stage from the multi-stage voice activity is further based on the received display data.
5. The method of claim 1 , wherein the multi-stage voice activity comprises an application-specific task for the application to which the voice input is directed.
6. The method of claim 1 , wherein the data processing hardware resides on the user device.
7. The method of claim 1 , wherein the operations further comprise biasing the language model by decreasing the respective non-zero probability score for each other n-gram not included in the identified representative subset of n-grams.
8. The method of claim 1 , wherein the language model comprises an n-gram language model.
9. The method of claim 1 , wherein the user device comprises a multimedia computing device.
10. The method of claim 1 , wherein the user device comprises a wearable computing device.
11. A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware causes the data processing hardware to perform operations comprising:
receiving a transcription request for a voice input captured by a user device, the transcription request comprising audio data and context data, the audio data indicating the voice input and the context data indicating:
an application to which the voice input is directed; and
a particular stage from a multi-stage voice activity corresponding to a series of user interactions related to a task of the application;
based on the context data, determining, from among multiple possible dialogs, a particular dialog corresponding to the particular stage from the multi-stage voice activity;
based on the particular dialog corresponding to the particular stage from the multi-stage dialog, filtering the audio data to only include audio data associated with the particular dialog;
identifying, from a set of n-grams, a representative subset of n-grams associated with the particular dialog corresponding to the particular stage from the multi-stage voice activity based on one or more prior stages from the multi-stage voice activity corresponding to one or more prior transcription requests, each n-gram of the set of n-grams comprising a respective non-zero probability score;
biasing a language model by increasing the respective non-zero probability score each n-gram of the identified representative subset of n-grams associated with the particular dialog; and
processing, using the biased language model, the filtered audio data to generate a transcription of the voice input.
12. The system of claim 11 , wherein the identified representative subset of n-grams associated with the particular dialog corresponding to the particular stage from the multi-stage voice activity is further based at least on n-grams in the representative subset of n-grams occurring frequently in historical voice inputs that correspond to the particular dialog.
13. The system of claim 11 , wherein the operations further comprise:
generating an initial transcription of the voice input using the language model prior to biasing the language model,
wherein determining the particular dialog corresponding to the particular stage from the multi-stage voice activity is further based on the initial transcription.
14. The system of claim 11 , wherein the operations further comprise:
receiving display data associated with a user interface of the user device at a time the voice input was captured by the user device,
wherein determining the particular dialog corresponding to the particular stage from the multi-stage voice activity is further based on the received display data.
15. The system of claim 11 , wherein the multi-stage voice activity comprises an application-specific task for the application to which the voice input is directed.
16. The system of claim 11 , wherein the data processing hardware resides on the user device.
17. The system of claim 11 , wherein the operations further comprise biasing the language model by decreasing the respective non-zero probability score for each other n-gram not included in the identified representative subset of n-grams.
18. The system of claim 11 , wherein the language model comprises an n-gram language model.
19. The system of claim 11 , wherein the user device comprises a multimedia computing device.
20. The system of claim 11 , wherein the user device comprises a wearable computing device.