Electronic device supporting speech recognition and operating method thereof
Various embodiments of the disclosure relate to an apparatus for supporting automatic speech recognition in an electronic device capable of performing natural language processing, and an operating method thereof. To this end, based on one or more partial texts recognized sequentially from an audio signal input from a microphone, the electronic device may identify an utterance type for the input audio signal. The electronic device may adaptively determine a hangover time, based on the utterance type, and may detect an end of an utterance of the audio signal by applying the determined hangover time.
1 . An electronic device comprising:
a microphone configured to convert and output a signal applied from the outside by an utterance into an electrical audio signal;
memory storing instructions; and
at least one processor electrically coupled to the microphone and the memory,
wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
identify an utterance type for an input audio signal based on one or more partial texts recognized sequentially from the audio signal input from the microphone;
adaptively determine a hangover time based on the identified utterance type; and
detect an end of the utterance for the audio signal by applying the hangover time,
wherein the utterance type for the input audio signal corresponds to one of a complete sentence, an incomplete sentence, and an ambiguous sentence,
wherein the complete sentence is a text in which an instruction instructing execution of an operation is identifiable based on natural language understanding, and the hangover time is set to be less than a reference hangover time when the identified utterance type corresponds to the complete sentence,
wherein the incomplete sentence is a text in which the instruction instructing execution of the operation is not identifiable based on the natural language understanding, and the hangover time is set to be greater than the reference hangover time when the identified utterance type corresponds to the incomplete sentence, and
wherein the ambiguous sentence is a text not classified as the complete sentence or the incomplete sentence, and the hangover time is set to be the reference hangover time when the identified utterance type corresponds to the ambiguous sentence.
2 . The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to determine that the end of the utterance is detected based on a pause being maintained for the hangover time.
3 . The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to at least one processor to decrease the hangover time by a degree corresponding to a speech rate of the audio signal upon identifying that the utterance type is the complete sentence using a speech recognition technique.
4 . The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to increase the hangover time by a degree corresponding to a speech rate of the audio signal upon identifying that the utterance type is the incomplete sentence using a speech recognition technique.
5 . The electronic device of claim 4 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to maintain the hangover time upon identifying that the utterance type is the ambiguous sentence using a speech recognition technique.
6 . The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to process a full text recognized from the input audio signal to be output as a machine language, based on a pause being maintained until the hangover time elapses.
7 . The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to consider characteristic data together with the identified utterance type to adaptively set the hangover time, wherein the characteristic data is a value indicating a degree for at least one of a speech rate, a pause time between utterances, a pause count, gender, age, region, and emotion.
8 . The electronic device of claim 7 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to accumulate the characteristic data in the memory using a deep-learning technique.
9 . The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
determine whether a partial text obtained on a real-time basis from the input audio signal or a combination of one or more partial texts obtained previously and the partial text obtained on the real-time basis is the complete sentence or the incomplete sentence using a bidirectional gated recurrent unit structure.
10 . The electronic device of claim 9 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: learn the complete sentence or the incomplete sentence using a deep-learning technique, and accumulate data based on the learning in the memory.
11 . A method of operating an electronic device, the method comprising:
identifying an utterance type for an input audio signal, based on one or more partial texts recognized sequentially from an audio signal comprising an electrical signal converted from a signal applied from the outside by an utterance;
adaptively determining a hangover time based on the identified utterance type; and
detecting an end of the utterance for the audio signal by applying the hangover time,
wherein the utterance type for the input audio signal corresponds to one of a complete sentence, an incomplete sentence, and an ambiguous sentence,
wherein the complete sentence is a text in which an instruction instructing execution of an operation is identifiable based on natural language understanding and the hangover time is set to be less than a reference hangover time when the identified utterance type corresponds to the complete sentence,
wherein the incomplete sentence is a text in which the instruction instructing execution of the operation is not identifiable based on the natural language understanding and the hangover time is set to be greater than the reference hangover time when the identified utterance type corresponds to the incomplete sentence, and
wherein the ambiguous sentence is a text not classified as the complete sentence or the incomplete sentence, and the hangover time is set to be the reference hangover time when the identified utterance type corresponds to the ambiguous sentence.
12 . The method of claim 11 , wherein the detecting of the end of the utterance includes determining that the end of the utterance is detected based on a pause being maintained for the hangover time.
13 . The method of claim 11 , wherein the adaptively determining of the hangover time comprises decreasing the hangover time by a degree corresponding to a speech rate of the audio signal upon identifying that the utterance type is the complete sentence using a speech recognition technique.
14 . The method of claim 11 , wherein the adaptively determining of the hangover time comprises increasing the hangover time by a degree corresponding to a speech rate of the audio signal upon identifying that the utterance type is the incomplete sentence using a speech recognition technique.
15 . The method of claim 11 , wherein the adaptively determining of the hangover time comprises maintaining the hangover time upon identifying that the utterance type is the ambiguous sentence using a speech recognition technique.
16 . The method of claim 11 , further comprising processing a full text recognized from the input audio signal to be output as a machine language, based on a pause being maintained until the hangover time elapses.
17 . The method of claim 11 , further comprising adaptively setting the hangover time by considering characteristic data together with the identified utterance type, wherein the characteristic data is a value indicating a degree for at least one of a speech rate, a pause time between utterances, a pause count, gender, age, region, and emotion.
18 . The method of claim 17 , further comprising accumulating the characteristic data in memory using a deep-learning technique.
19 . The method of claim 11 ,
wherein the identifying of the utterance type comprises determining whether a partial text obtained on a real-time basis from the input audio signal or a combination of one or more partial texts obtained previously and the partial text obtained on the real-time basis is the complete sentence or the incomplete sentence using a bidirectional gated recurrent unit structure.
20 . The method of claim 19 , further comprising learning the complete sentence or the incomplete sentence using a deep-learning technique, and accumulating data based on the learning in memory.