Speech recognition hypothesis generation according to previous occurrences of hypotheses terms and/or contextual data
Implementations set forth herein relate to speech recognition techniques for handling variations in speech among users (e.g. due to different accents) and processing features of user context in order to expand a number of speech recognition hypotheses when interpreting a spoken utterance from a user. In order to adapt to an accent of the user, terms common to multiple speech recognition hypotheses can be filtered out in order to identify inconsistent terms apparent in a group of hypotheses. Mappings between inconsistent terms can be stored for subsequent users as term correspondence data. In this way, supplemental speech recognition hypotheses can be generated and subject to probability-based scoring for identifying a speech recognition hypothesis that most correlates to a spoken utterance provided by a user. In some implementations, prior to scoring, hypotheses can be supplemented based on contextual data, such as on-screen content and/or application capabilities.
1 . A method for performing speech recognition on a spoken utterance from a user, the method implemented by one or more processors and comprising:
processing, at a speech recognition engine of a computing device, audio data corresponding to the spoken utterance;
generating, based on processing the audio data and by the speech recognition engine of the computing device, a plurality of current speech recognition hypotheses,
wherein each current speech recognition hypothesis of the plurality of current speech recognition hypotheses includes corresponding terms that are predicted to correspond to original natural language content of the spoken utterance from the user;
identifying term correspondence data that characterizes relationships between previous terms provided in previous speech recognition hypotheses generated based on previous spoken utterances from the user;
determining, based on the term correspondence data, that a given term, of at least a given current hypothesis of the plurality of current speech recognition hypotheses:
is included in the term correspondence data, and
corresponds to a related term of at least a given previous hypothesis of the previous speech recognition hypotheses, in the term correspondence data, that is not included in any of the plurality of current speech recognition hypotheses based at least in part on the given term sharing a common position in the given current hypothesis with the related term in the given previous speech recognition hypothesis;
based on determining that the given term is included in the term correspondence data and corresponds to the related term that is not included in any of the plurality of current speech recognition hypotheses:
generating, via the speech recognition engine of the computing device, a supplemental current speech recognition hypothesis that conforms to the given current hypothesis, but replaces the given term with the related term; and
selecting the supplemental current speech recognition hypothesis as an actual speech recognition result; and
in response to the selecting, causing the computing device to render an output based on the supplemental current speech recognition hypothesis.
2 . The method of claim 1 , wherein selecting the supplemental current speech recognition hypothesis as the actual speech recognition result is based on contextual data that characterizes a context in which the user provided the spoken utterance.
3 . The method of claim 2 , wherein the contextual data characterizes graphical content being rendered at a graphical user interface of the computing device when the user provided the spoken utterance.
4 . The method of claim 2 , wherein the contextual data characterizes one or more applications that are accessible via the computing device.
5 . The method of claim 4 , wherein selecting the supplemental current speech recognition hypothesis as the actual speech recognition result based on contextual data comprises:
selecting the supplemental current speech recognition hypothesis based on determining that the supplemental current speech recognition hypothesis corresponds to an action that is capable of being initialized via the one or more applications that are accessible via the computing device.
6 . The method of claim 1 , wherein the given term corresponds to the related term, in the term correspondence data, based on the given term occurring in a first previous speech recognition hypothesis generated based on a first previous spoken utterance from the user and the related term occurring in a second previous speech recognition hypothesis generated based on the same first previous spoken utterance.
7 . The method of claim 1 , further comprising, prior to processing the audio data:
generating, in the term correspondence data, the correspondence between the given term and the related term, wherein generating the correspondence is in response to the given term and the related term both being included, in one or more corresponding previous speech recognition hypotheses, for one or more same of the previous spoken utterances.
8 . A computing device comprising:
one or more microphones;
memory storing term correspondence data that characterizes relationships between previous terms provided in previous speech recognition hypotheses generated based on previous spoken utterances from a user; and
one or more processors configured to:
process, via a speech recognition engine, audio data that is captured via the one or more microphones and that captures a spoken utterance of the user;
generate, based on processing the audio data and via the speech recognition engine, a plurality of current speech recognition hypotheses,
wherein each current speech recognition hypothesis of the plurality of current speech recognition hypotheses includes corresponding terms that are predicted to correspond to original natural language content of the spoken utterance from the user;
determine, based on the term correspondence data, that a given term, of at least a given current hypothesis of the plurality of current speech recognition hypotheses:
is included in the term correspondence data, and
corresponds to a related term of at least a given previous hypothesis of the previous speech recognition hypotheses, in the term correspondence data, that is not included in any of the plurality of current speech recognition hypotheses based at least in part on the given term sharing a common position in the given current hypothesis with the related term in the given previous speech recognition hypothesis;
based on determining that the given term is included in the term correspondence data and corresponds to the related term that is not included in any of the plurality of current speech recognition hypotheses:
generate, via the speech recognition engine, a supplemental current speech recognition hypothesis that conforms to the given current hypothesis, but replaces the given term with the related term; and
select the supplemental current speech recognition hypothesis as an actual speech recognition result; and
in response to the selecting, cause an output to be rendered based on the supplemental current speech recognition hypothesis.
9 . The computing device of claim 8 , wherein in selecting the supplemental current speech recognition hypothesis as the actual speech recognition result one or more of the processors are to select the supplemental current speech recognition hypothesis as the actual speech recognition result based on contextual data that characterizes a context in which the user provided the spoken utterance.
10 . The computing device of claim 9 , wherein the contextual data characterizes graphical content being rendered at a graphical user interface of the computing device when the user provided the spoken utterance.
11 . The computing device of claim 9 , wherein the contextual data characterizes one or more applications that are accessible via the computing device.
12 . The computing device of claim 11 , wherein in selecting the supplemental current speech recognition hypothesis as the actual speech recognition result based on contextual data one or more of the processors are to:
select the supplemental current speech recognition hypothesis based on determining that the supplemental current speech recognition hypothesis corresponds to an action that is capable of being initialized via the one or more applications that are accessible via the computing device.
13 . The computing device of claim 8 , wherein the given term corresponds to the related term, in the term correspondence data, based on the given term occurring in a first previous speech recognition hypothesis generated based on a first previous spoken utterance from the user and the related term occurring in a second previous speech recognition hypothesis generated based on the same first previous spoken utterance.
14 . The computing device of claim 8 , wherein one or more of the processors are further to, prior to processing the audio data:
generate, in the term correspondence data, the correspondence between the given term and the related term, wherein one or more of the processors are to generate the correspondence in response to the given term and the related term both being included, in one or more corresponding previous speech recognition hypotheses, for one or more same of the previous spoken utterances.