SYSTEM AND METHOD FOR ENRICHING TEXT-TO-SPEECH SYNTHESIS WITH AUTOMATIC DIALOG ACT TAGS
Disclosed herein are systems, methods, and non-transitory computer-readable storage media for modifying the prosody of synthesized speech based on an associated speech act. A system configured according to the method embodiment (1) receives text, (2) performs an analysis of the text to determine and assign a speech act label to the text, and (3) converts the text to speech, where the speech prosody is based on the speech act label. The analysis performed compares the text to a corpus of previously tagged utterances to find a close match, determines a confidence score from a correlation of the text and the close match, and, if the confidence score is above a threshold value, retrieving the speech act label of the close match and assigning it to the text.
1 . A method comprising:
receiving text;
analyzing, via a processor, the text to determine and assign a speech act label to the text, wherein analyzing comprises:
comparing the text to a corpus of previously labeled utterances to find a closest match;
determining a confidence score from a correlation of the text and the closest match; and
if the confidence score is above a threshold value, retrieving the speech act label from the closest match and assigning the speech act label to the text; and
converting the text to speech, wherein the speech has a prosody based at least in part on the speech act label.
2 . The method of claim 1 , wherein the text is assigned a plurality of speech act labels.
3 . The method of claim 1 , wherein analyzing occurs within a dialog management module.
4 . The method of claim 1 , wherein analyzing further comprises:
assigning additional speech labels based on at least one of dialect, accent, message repetition, and text theme.
5 . The method of claim 1 , wherein the prosody describes at least one of tone, intonation, rhythm, focus, syllable length, loudness, pitch, formant, and lexical stress.
6 . The method of claim 1 , wherein the prosody matches a stored prosody found in the corpus.
7 . The method of claim 1 , further comprising:
outputting the speech to a user.
8 . A system comprising:
a processor; and
a storage device storing instructions for controlling the processor to perform steps comprising:
receiving text;
analyzing the text to determine and assign a speech act label to the text, wherein analyzing comprises:
comparing the text to a corpus of previously labeled utterances to find a closest match;
determining a confidence score from a correlation of the text and the closest match; and
if the confidence score is above a threshold value, retrieving the speech act label from the closest match and assigning the speech act label to the text; and
converting the text to speech, wherein the speech has a prosody based at least in part on the speech act label.
9 . The system of claim 8 , wherein the text is assigned a plurality of speech act labels.
10 . The system of claim 8 , wherein analyzing occurs within a dialog management module.
11 . The system of claim 8 , analyzing further comprises:
assigning additional speech labels based on at least one of dialect, accent, message repetition, and text theme.
12 . The system of claim 8 , wherein the prosody describes at least one of tone, intonation, rhythm, focus, syllable length, loudness, pitch, formant, and lexical stress.
13 . The system of claim 8 , wherein the prosody matches a stored prosody found in the corpus.
14 . The system of claim 8 , further comprising:
outputting the speech to a user.
15 . A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform steps comprising:
receiving text;
analyzing the text to determine and assign a speech act label to the text, wherein analyzing comprises:
comparing the text to a corpus of previously labeled utterances to find a closest match;
determining a confidence score from a correlation of the text and the closest match; and
if the confidence score is above a threshold value, retrieving the speech act label from the closest match and assigning the speech act label to the text; and
converting the text to speech, wherein the speech has a prosody based at least in part on the speech act label.
16 . The non-transitory computer-readable storage medium of claim 15 , wherein the text is assigned a plurality of speech act labels.
17 . The non-transitory computer-readable storage medium of claim 15 , wherein analyzing occurs within a dialog management module.
18 . The non-transitory computer-readable storage medium of claim 15 , wherein analyzing further comprises:
assigning additional speech labels based on at least one of dialect, accent, message repetition, and text theme.
19 . The non-transitory computer-readable storage medium of claim 15 , wherein the prosody describes at least one of tone, intonation, rhythm, focus, syllable length, loudness, pitch, formant, and lexical stress.
20 . The non-transitory computer-readable storage medium of claim 15 , wherein the prosody matches a stored prosody found in the corpus.