Context-based detection of end-point of utterance
Detecting an end-point of user's voice command or utterance with high accuracy is critical in automatic speech recognition (ASR)-based human machine interface. If an ASR system incorrectly detects an end-point of utterance and transmits this incomplete sentence to other processing blocks for further processing, it is likely the processed result would lead to incorrect interpretation. A method includes selecting a first semantic network based on context of the audio signal and more accurately detecting the end-point of user's utterance included in the audio signal based on the first semantic network and also based on at least one timeout threshold associated with the first semantic network.
1 . A method for processing an audio signal by an electronic device, the method comprising:
receiving, by an automatic speech recognition (ASR) module, the audio signal representing an utterance of a user;
selecting, by the ASR module, a first semantic network based on context of the audio signal, wherein the first semantic network comprises a plurality of slots, each slot of the plurality of slots associated with a set of words;
generating, based on automatic speech recognition processing performed on a first portion of the audio signal by the ASR module, a first ASR output that indicates one or more words identified in the utterance and an end time of the utterance;
associating, by the ASR module and based on the first ASR output, the one or more words with one or more slots of the plurality of slots; and
in response to a determination that passage of time after the end time of the utterance exceeds a timeout threshold amount of time associated with a last filled slot of the one or more slots:
determining an amount of time of time to adjust the timeout threshold amount of time, wherein a duration of the amount of time is determined based on data for the user; and
augmenting the timeout threshold amount of time associated with the last filled slot by the determined amount of time to adjust the timeout threshold, wherein the data is associated with the first semantic network.
2 . The method of claim 1 , wherein selecting the first semantic network based on the context of the audio signal comprises determining the context of the audio signal based on at least a portion of the first ASR output.
3 . The method of claim 2 , wherein selecting the first semantic network based on the context of the audio signal further comprises selecting the first semantic network from a plurality of semantic networks based on the context.
4 . The method of claim 3 , wherein at least one semantic network of the plurality of semantic networks corresponds to a recursive transition network (RTN).
5 . The method of claim 1 , further comprising:
parsing the first ASR output based on a predetermined grammar to generate parsed first ASR output;
determining, based on the parsed first ASR output, the first ASR output corresponds to an incomplete sentence; and
comparing the parsed first ASR output with the first semantic network.
6 . The method of claim 5 , wherein comparing the parsed first ASR output with the first semantic network comprises:
identifying a first portion of the parsed first ASR output that corresponds to a first slot among the plurality of slots associated with the first semantic network; and
identifying a second portion of the parsed first ASR output that does not correspond to a second slot among the plurality of slots associated with the first semantic network.
7 . An apparatus for processing an audio signal, the apparatus comprising:
at least one processor; and
a memory coupled to the at least one processor, the memory including instructions executable to cause the at least one processor to:
receive, by an automatic speech recognition (ASR) module, the audio signal representing an utterance of a user;
select a first semantic network based on context of the audio signal, wherein the first semantic network comprises a plurality of slots, each slot of the plurality of slots associated with a set of words;
generate, based on automatic speech recognition processing performed on a first portion of the audio signal by the ASR module, a first ASR output that indicates one or more words identified in the utterance and an end time of the utterance;
associate the one or more words of the first ASR output with one or more slots of the plurality of slots; and
in response to a determination that passage of time after the end time of the utterance exceeds a timeout threshold amount of time associated with a last filled slot of the one or more slots:
determine an amount of time of time to adjust the timeout threshold amount of time, wherein a duration of the amount of time is determined based on data for the user; and
augment the timeout threshold amount of time associated with the last filled slot by the determined amount of time to adjust the timeout threshold, wherein the data is associated with the first semantic network.
8 . The apparatus of claim 7 , wherein, to select the first semantic network, the at least one processor is configured to determine the context of the audio signal based on at least a portion of the first ASR output.
9 . The apparatus of claim 7 , wherein the at least one processor is configured to select the first semantic network by selecting the first semantic network from a plurality of semantic networks based on the context.
10 . The apparatus of claim 7 , wherein:
the data for the user includes a history of prior utterances of the user, the prior utterances of the user occurred prior to the utterance of the user; and
each slot of the plurality of slots corresponds to a threshold time for the slot that indicates a time span after the slot is filled and during which one or more additional words are detectable in the audio signal before a timeout determination.
11 . An apparatus for processing an audio signal, the apparatus comprising:
means for receiving the audio signal representing an utterance of a user;
means for selecting a first semantic network based on context of the audio signal, wherein the first semantic network comprises a plurality of slots, each slot of the plurality of slots associated with a set of words;
means for generating, based on automatic speech recognition processing performed on a first portion of the audio signal, a first ASR output, the first ASR output including one or more words identified in the utterance and an end time of the utterance;
means for associating one or more words of the first ASR output with one or more slots of the plurality of slots;
means for determining whether a passage of time after the end time of the utterance exceeds a timeout threshold amount of time associated with a last filled slot of the one or more slots;
means for determining an amount of time of time to adjust the timeout threshold amount of time, wherein a duration of the amount of time is determined based on data for the user; and
means for augmenting the timeout threshold amount of time associated with the last filled slot by the determined amount of time to adjust the timeout threshold in response to a determination by the means for determining that the passage of time after the end time of the utterance exceeds the timeout threshold amount of time, wherein the data is associated with the first semantic network.
12 . The apparatus of claim 11 , wherein the means for selecting the first semantic network based on the context of the audio signal comprises means for determining the context of the audio signal based on at least a portion of the first ASR output.
13 . A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform operations comprising:
receiving, by an automatic speech recognition (ASR) module, an audio signal representing an utterance of a user;
selecting a first semantic network based on context of the audio signal, wherein the first semantic network comprises a plurality of slots, each slot of the plurality of slots associated with a set of words;
generating, based on automatic speech recognition processing performed on a first portion of the audio signal by the ASR module, a first ASR output that indicates one or more words identified in the utterance and an end time of the utterance;
associating the one or more words of the first ASR output with one or more slots of the plurality of slots; and
in response to a determination that passage of time after the end time of the utterance exceeds a timeout threshold amount of time associated with a last filled slot of the one or more slots:
determining an amount of time of time to adjust the timeout threshold amount of time, wherein a duration of the amount of time is determined based on data for the user; and
augmenting, by the ASR module, the timeout threshold amount of time associated with the last filled slot by the determined amount of time to adjust the timeout threshold, wherein the data is associated with the first semantic network.
14 . The non-transitory computer-readable medium of claim 13 , wherein selecting the first semantic network based on the context of the audio signal comprises determining the context of the audio signal based on at least a portion of the first ASR output.
15 . The non-transitory computer-readable medium of claim 13 , wherein selecting the first semantic network based on the context of the audio signal further comprises selecting the first semantic network from a plurality of semantic networks based on the context.
16 . The apparatus of claim 7 , wherein the at least one processor is configured to determine, prior to expiration of the augmented timeout threshold amount of time, whether a word of a second portion of the audio signal corresponds to at least one slot of the plurality of slots that is after the one or more slots.
17 . The apparatus of claim 16 , wherein the at least one processor is configured to:
perform automatic speech recognition processing on the first portion of the audio signal to generate the first ASR output; and
perform automatic speech recognition processing on the second portion of the audio signal to generate a second ASR output that indicates the word.
18 . The apparatus of claim 7 , wherein:
a first slot of the plurality of slots is associated with a first threshold time having a first value,
a second slot of the plurality of slots is associated with a second threshold time having a second value,
the first value different from the second value; and
the plurality of slots include compulsory slots, wherein each compulsory slot is configured to be filled for a determination that the utterance corresponds to a complete sentence.
19 . The apparatus of claim 18 , wherein:
the timeout threshold amount of time associated with the last filled slot is augmented in response to:
the determination that the passage of time after the end time of the utterance exceeds the timeout threshold amount of time associated with a last filled slot of the one or more slots; and
a determination that at least one compulsory slot of the compulsory slots is unfilled; and
the plurality of slots includes at least one optional slot, wherein the determination that the utterance corresponds to a complete sentence is determinable independent of the optional slot being filled.
20 . The apparatus of claim 7 , wherein:
the at least one processor is configured to, for each slot of the plurality of slots, determine, based on the context, the set of words associated with the slot; and
wherein:
a first slot of the plurality of slots is associated with a first set of one or more words of the set of words, the first set of one or more words based on the context;
a second slot of the plurality of slots is associated with a second set of one or more words of the set of words, the second set of one or more words based on the context; and
the first set of one or more words is different from the second set of one or more words, and wherein the set of words is based on the context.