Acoustic model training using corrected terms
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for speech recognition. One of the methods includes receiving first audio data corresponding to an utterance; obtaining a first transcription of the first audio data; receiving data indicating (i) a selection of one or more terms of the first transcription and (ii) one or more of replacement terms; determining that one or more of the replacement terms are classified as a correction of one or more of the selected terms; in response to determining that the one or more of the replacement terms are classified as a correction of the one or more of the selected terms, obtaining a first portion of the first audio data that corresponds to one or more terms of the first transcription; and using the first portion of the first audio data that is associated with the one or more terms of the first transcription to train an acoustic model for recognizing the one or more of the replacement terms.
1 . A computer-implemented method executed on data processing hardware that causes the data processing hardware to perform operations comprising:
receiving a voice input from a user of a user device;
processing, using a trained voice recognition engine, audio data representing the voice input to generate a first complete transcription of the voice input and one or more complete alternative transcriptions of the voice input, the voice recognition engine trained by a training process that comprises:
obtaining a plurality of training samples, each training sample comprising corresponding audio data representing a corresponding utterance and a corresponding transcription of the corresponding utterance; and
for each training sample, training the voice recognition engine to predict, based on the corresponding audio data representing the corresponding utterance, the corresponding transcription;
displaying, in a graphical user interface of the user device, the first complete transcription of the voice input;
displaying, in the graphical user interface, the one or more complete alternative transcriptions of the voice input;
receiving, via the graphical user interface, a selection indication indicating a user selection of a selected transcription of the first complete transcription or the one or more complete alternative transcriptions displayed in the graphical user interface to provide a corrected transcription of the voice input;
in response to receiving the selection indication indicating user selection of the selected transcription, displaying, in the graphical user interface, one or more links to resources determined to be responsive to the selected transcription; and
receiving, via the graphical user interface, another selection indication indicating user selection of one of the one or more links displayed in the graphical user interface to cause the graphical user interface to present a new screen based on the user selection of the one of the one or more links.
2 . The method of claim 1 , wherein the operations further comprise, after receiving the voice input, displaying, in the graphical user interface, one or more other links to resources determined to be responsive to the first complete transcription.
3 . The method of claim 1 , wherein the operations further comprise, prior to receiving the selection indication indicating the user selection of the selected transcription, receiving, at the user device, a selection indication indicating a user selection in the graphical user interface to correct the first complete transcription.
4 . The method of claim 3 , wherein the operations further comprise, in response to receiving the selection indication indicating the user selection in the graphical user interface to correct the first complete transcription, displaying a keyboard in the graphical user interface.
5 . The method of claim 1 , wherein the operations further comprise providing the selected transcription to a search engine, the search engine configured to generate search results that are determined to be responsive to the selected transcription.
6 . The method of claim 5 , wherein the generated search results comprise a particular resource determined by the search engine to be responsive to a type of request in the voice input.
7 . The method of claim 1 , wherein the one or more links to resources determined to be responsive to the selected transcription displayed in the graphical user interface comprise one or more suggested terms.
8 . The method of claim 1 , wherein the first complete transcription comprises a mis-transcribed term that is replaced in at least one of the one or more complete alternative transcriptions.
9 . The method of claim 8 , wherein the at least one of the one or more complete alternative transcriptions spells out a corresponding replacement term for the mis-transcribed term.
10 . The method of claim 9 , wherein the operations further comprise isolating at least a portion of the voice input as training data for training a model to recognize the corresponding replacement term.
11 . A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware and storing instructions that, when executed by the data processing hardware, cause the data processing hardware to perform operations comprising:
receiving a voice input from a user of a user device;
processing, using a trained voice recognition engine, audio data representing the voice input to generate a complete first transcription of the voice input and one or more complete alternative transcriptions of the voice input, the voice recognition engine trained by a training process that comprises:
obtaining a plurality of training samples, each training sample comprising corresponding audio data representing a corresponding utterance and a corresponding transcription of the corresponding utterance; and
for each training sample, training the voice recognition engine to predict, based on the corresponding audio data representing the corresponding utterance, the corresponding transcription;
displaying, in a graphical user interface of the user device, the first complete transcription of the voice input;
displaying, in the graphical user interface, the one or more complete alternative transcriptions of the voice input;
receiving, via the graphical user interface, a selection indication indicating a user selection of a selected transcription of the first complete transcription or the one or more complete alternative transcriptions displayed in the graphical user interface to provide a corrected transcription of the voice input;
in response to receiving the selection indication indicating user selection of the selected transcription, displaying, in the graphical user interface, one or more links to resources determined to be responsive to the selected transcription; and
receiving, via the graphical user interface, another selection indication indicating user selection of one of the one or more links displayed in the graphical user interface to cause the graphical user interface to present a new screen based on the user selection of the one of the one or more links.
12 . The system of claim 11 , wherein the operations further comprise, after receiving the voice input, displaying, in the graphical user interface, one or more other links to resources determined to be responsive to the first complete transcription.
13 . The system of claim 11 , wherein the operations further comprise, prior to receiving the selection indication indicating the user selection of the selected transcription, receiving, at the user device, a selection indication indicating a user selection in the graphical user interface to correct the first complete transcription.
14 . The system of claim 13 , wherein the operations further comprise, in response to receiving the selection indication indicating the user selection in the graphical user interface to correct the first complete transcription, displaying a keyboard in the graphical user interface.
15 . The system of claim 11 , wherein the operations further comprise providing the selected transcription to a search engine, the search engine configured to generate search results that are determined to be responsive to the selected transcription.
16 . The system of claim 15 , wherein the generated search results comprise a particular resource determined by the search engine to be responsive to a type of request in the voice input.
17 . The system of claim 11 , wherein the one or more links to resources determined to be responsive to the selected transcription displayed in the graphical user interface comprise one or more suggested terms.
18 . The system of claim 11 , wherein the first complete transcription comprises a mis-transcribed term that is replaced in at least one of the one or more complete alternative transcriptions.
19 . The system of claim 18 , wherein the at least one of the one or more complete alternative transcriptions spells out a corresponding replacement term for the mis-transcribed term.
20 . The system of claim 19 , wherein the operations further comprise isolating at least a portion of the voice input as training data for training a model to recognize the corresponding replacement term.