Information processing apparatus that fades system utterance in response to interruption
An apparatus and method are capable of controlling the output of the system utterance upon the occurrence of barge-in utterance and enabling a smooth interactive between a user and the system. Fade processing is applied to lower at least one of volume, a speech rate, or a pitch (voice pitch) of system utterance from a starting time of the barge-in utterance acting as the user interruption utterance during executing the system utterance. Even after the completion of the fade processing, the output state upon completing the fade processing is maintained. In a case where the system utterance level is equal to or less than the predefined threshold during the fade processing, the system utterance is displayed on a display unit. One of stop, continuation, and rephrasing is executed based on an intention of the barge-in utterance and whether an important word is included in in the system utterance.
1. An information processing apparatus comprising:
a microphone;
a speaker;
a display; and
processing circuitry configured to
perform a speech synthesis process to change text information to speech information,
control the speaker to output the speech information as system utterance,
receive, via the microphone, barge-in utterance, which is interruption utterance by a user,
in response to receiving the barge-in utterance while the system utterance is outputted by the speaker, start fade processing of lowering at least one of volume, a speech rate, or a pitch of the system utterance by controlling the speaker, determine whether the at least one of the volume, the speech rate, or the pitch of the system utterance has become equal to or less than a predefined threshold by the fade processing, and
in response determining that the at least one of the volume, the speech rate, or the pitch of the system utterance has become equal to or less than the predefined threshold by the fade processing, control the display to display the text information corresponding to the system utterance in such a manner to distinguish between a part of the text information that has been outputted by the speaker and another part of the text information that has not yet been outputted by the speaker, wherein
the processing circuitry is further configured to
compare the text information and the barge-in utterance,
identify a contradictory part between the text information and the barge-in utterance, and
control the display to highlight the contradictory part in the text information.
2. The information processing apparatus according to claim 1 , wherein
the processing circuitry has a voice activity detection (VAD) function, and starts the fade processing from a starting time of the barge-in utterance decided on a basis of the VAD function.
3. The information processing apparatus according to claim 1 , wherein the processing circuitry continues the fade processing for a predetermined specified time and, after a lapse of the specified time, maintains an output state of the system utterance at a time of completion of the fade processing.
4. The information processing apparatus according to claim 1 , wherein the threshold corresponds to a level at which the system utterance is difficult for a user to hear.
5. The information processing apparatus according to claim 1 , wherein the processing circuitry determines
whether or not the system utterance is necessary to be stopped on a basis of the barge-in utterance's intention and, in a case of determining that the system utterance is necessary to be stopped, stops the system utterance.
6. The information processing apparatus according to claim 1 , wherein the processing circuitry stops the system utterance in a case of determining that a system response speech output is necessary as a system action corresponding to the barge-in utterance's intention.
7. The information processing apparatus according to claim 1 , wherein the processing circuitry:
determines whether or not the system utterance is necessary to be stopped on a basis of the barge-in utterance's intention, and
performs, in a case of determining that the system utterance is unnecessary to be stopped, processing of determining which of
(a) returning the system utterance to a state before the fade processing and continuing an output, or
(b) returning the system utterance to the state before the fade processing and executing rephrasing processing is to be executed.
8. The information processing apparatus according to claim 7 , wherein the processing circuitry:
determines whether or not an important word is included in the system utterance after the start of the fade processing within a period from a start to an end of the barge-in utterance, and
performs,
in a case where the important word is not included,
(a) returning to the state before the fade processing and continuing an output, and in a case where the important word is included,
(b) returning to the state before the fade processing and executing the rephrasing processing.
9. The information processing apparatus according to claim 8 , wherein the processing circuitry,
in a case of performing (b) returning to the state before the fade processing and executing the rephrasing processing,
starts the rephrasing processing from an utterance position of a phrase including an initial important word of the system utterance after the start of the fade processing.
10. The information processing apparatus according to claim 8 , wherein the important word is one of words indicating a proper noun, date, time, a place, information of an amount of money, numerals, an object, and an objective case.
11. The information processing apparatus according to claim 1 , wherein the processing circuitry,
in response determining that the at least one of the volume, the speech rate, or the pitch of the system utterance has become equal to or less than the predefined threshold by the fade processing, control the display to display the text information corresponding to the system utterance, and the barge-in utterance.
12. The information processing apparatus according to claim 1 , wherein the processing circuitry is configured to:
in response determining that the at least one of the volume, the speech rate, or the pitch of the system utterance has become equal to or less than the predefined threshold by the fade processing, control the display to display the text information corresponding to the system utterance in such a manner to highlight the another part of the text information that has not yet been outputted by the speaker, and not highlight the part of the text information that has been outputted by the speaker.
13. The information processing apparatus according to claim 12 , wherein the processing circuitry is configured to highlight the another part of the text information that has not yet been outputted by the speaker, by changing the another part of the text information to be bold.
14. The information processing apparatus according to claim 1 , wherein the processing circuitry is configured to control the display to highlight the contradictory part in the text information by underlining the contradictory part.
15. The information processing apparatus according to claim 1 , wherein the processing circuitry is configured to
compare the another part of the text information that has not yet been outputted by the speaker and the barge-in utterance,
identify a contradictory part between the another part of the text information that has not yet been outputted by the speaker and the barge-in utterance, and
control the display to highlight the contradictory part in the another part of the text information that has not yet been outputted by the speaker.
16. An information processing method executed in an information processing apparatus, the method comprising:
performing a speech synthesis process to change text information to speech information,
controlling a speaker of the information processing apparatus to output the speech information as system utterance,
receiving, via a microphone of the information processing apparatus, barge-in utterance, which is interruption utterance by a user,
in response to receiving the barge-in utterance while the system utterance is outputted by the speaker, starting, using processing circuitry, fade processing of lowering at least one of volume, a speech rate, or a pitch of the system utterance by controlling the speaker,
determining whether the at least one of the volume, the speech rate, or the pitch of the system utterance has become equal to or less than a predefined threshold by the fade processing, and
in response determining that the at least one of the volume, the speech rate, or the pitch of the system utterance has become equal to or less than the predefined threshold by the fade processing, controlling a display of the information processing apparatus to display the text information corresponding to the system utterance in such a manner to distinguish between a part of the text information that has been outputted by the speaker and another part of the text information that has not yet been outputted by the speaker, wherein
the method further comprises
comparing the text information and the barge-in utterance,
identifying a contradictory part between the text information and the barge-in utterance, and
controlling the display to highlight the contradictory part in the text information.
17. A non-transitory computer readable medium including executable instructions, which when executed by a computer cause the computer to execute an information processing method executed in an information processing apparatus, the method comprising:
performing a speech synthesis process to change text information to speech information,
controlling a speaker of the information processing apparatus to output the speech information as system utterance,
receiving, via a microphone of the information processing apparatus, barge-in utterance, which is interruption utterance by a user,
in response to receiving the barge-in utterance while the system utterance is outputted by the speaker, starting fade processing of lowering at least one of volume, a speech rate, or a pitch of the system utterance by controlling the speaker,
determining whether the at least one of the volume, the speech rate, or the pitch of the system utterance has become equal to or less than a predefined threshold by the fade processing, and
in response determining that the at least one of the volume, the speech rate, or the pitch of the system utterance has become equal to or less than the predefined threshold by the fade processing, controlling a display of the information processing apparatus to display the text information corresponding to the system utterance in such a manner to distinguish between a part of the text information that has been outputted by the speaker and another part of the text information that has not yet been outputted by the speaker, wherein
the method further comprises
comparing the text information and the barge-in utterance,
identifying a contradictory part between the text information and the barge-in utterance, and
controlling the display to highlight the contradictory part in the text information.