Method and system for generating synthesized speech based on human recording
View Patent ↗A method and system that incorporates human recording with a TTS system to generate synthesized speech with high quality by searching over a database of pre-recorded utterances to select an utterance best matching text content to be synthesized into speech; dividing the best-matched utterance into a plurality of segments to generate remaining segments that are the same as corresponding parts of the text content and difference segments that are different from corresponding parts of the text content; synthesizing speech for the parts of the text content corresponding to the difference segments; and splicing the synthesized speech segments with the remaining segments of the best-matched utterance.
1. A computer-implemented method for generating synthesized speech from input text, the method comprising:
selecting a best-matched pre-recorded utterance from a plurality of pre-recorded utterances, wherein the selecting is based, at least in part, on a degree of matching between the input text and texts associated with the plurality of pre-recorded utterances;
dividing the best-matched pre-recorded utterance into a plurality of segments comprising remaining segments that match corresponding parts of the input text and difference segments that do not match corresponding parts of the input text;
synthesizing speech for parts of the input text corresponding to the difference segments in the selected best-matched pre-recorded utterance to generate synthesized speech segments; and
splicing the synthesized speech segments of the parts of the input text corresponding to the difference segments with the remaining segments of the selected best-matched pre-recorded utterance to generate the synthesized speech for the input text.
2. The method according to claim 1 , wherein selecting a best-matched pre-recorded utterance comprises:
calculating an edit-distance between the input text and each of the plurality of pre-recorded utterances;
selecting the pre-recorded utterance with a minimum edit-distance as the best-matched pre-recorded utterance; and
determining at least one edit operation for converting the best-matched pre-recorded utterance into the synthesized speech for the input text.
3. The method according to claim 2 , wherein calculating an edit-distance is performed as follows:
E
(
i
,
j
)
=
min
{
E
(
i
-
1
,
j
-
1
)
+
Dis
(
s
i
,
t
j
)
E
(
i
,
j
-
1
)
+
Del
(
t
j
)
E
(
i
-
1
,
j
)
+
Ins
(
s
i
)
}
where S=s 1 . . . s i . . . s N represents a sequence of words in the pre-recorded utterance, T=t 1 . . . t j . . . t M represents a sequence of words in the input text, E(i, j) represents an edit-distance for converting, s 1 . . . s i into t 1 . . . t j , Dis(s i , t j ) represents a substitution penalty when replacing word s i in the pre-recorded utterance with word t j in the input text, Ins(s i ) represents an insertion penalty for inserting s i and Del(t j ) represents a deletion penalty for deleting t j .
4. The method according to claim 2 , wherein determining at least one edit operation comprises:
determining at least one editing location and at least one corresponding editing type.
5. The method according to claim 4 , wherein dividing the best-matched pre-recorded utterance into a plurality of segments comprises:
according to the determined at least one editing location, chopping out at least one edit segment to be edited from the best-matched pre-recorded utterance, wherein the include the at least one edit segment.
6. A system for generating synthesized speech for input text, the system comprising:
at least one storage device comprising a plurality of pre-recorded utterances; and
at least one computer configured to:
select a best-matched pre-recorded utterance from a plurality of pre-recorded utterances, wherein the selecting is based, at least in part, on a degree of matching between the input text and texts associated with the plurality of pre-recorded utterances;
divide the best-matched pre-recorded utterance into a plurality of segments comprising remaining segments that match corresponding parts of the input text and difference segments that do not match corresponding parts of the input text;
synthesize speech for parts of the input text corresponding to the difference segments in the selected best-matched pre-recorded utterance to generate synthesized speech segments; and
splice the synthesized speech segments with the remaining segments to generate synthesized speech for the input text.
7. The system according to claim 6 , wherein the at least one computer is further configured to:
calculate an edit-distance between the input text and each of the plurality of pre-recorded utterances in the at least one storage device;
select the pre-recorded utterance with minimum edit-distance as the best-matched utterance; and
determine at least one edit operation for converting the best-matched pre-recorded utterance into the synthesized speech for the input text.
8. The system according to claim 7 , wherein the edit-distance is calculated as follows:
E
(
i
,
j
)
=
min
{
E
(
i
-
1
,
j
-
1
)
+
Dis
(
s
i
,
t
j
)
E
(
i
,
j
-
1
)
+
Del
(
t
j
)
E
(
i
-
1
,
j
)
+
Ins
(
s
i
)
}
where S=s 1 . . . s i . . . s N represents a sequence of words in the pre-recorded utterance, T=t 1 . . . t j . . . t M represents a sequence of words in the input text, E(i, j) represents an edit-distance for converting, s 1 . . . s i into t 1 . . . t j , Dis(s i , t j ) represents a substitution penalty when replacing word s i in the pre-recorded utterance with word t j in the input text, Ins(s i ) represents an insertion penalty for inserting s i and Del(t j ) represents a deletion penalty for deleting t j .
9. The system according to claim 7 , wherein determining at least one edit operation comprises determining at least one editing location and at least one corresponding editing type.
10. The system according to claim 9 , wherein the at least one computer is further configured to:
chop out at least one edit segment to be edited from the best-matched pre-recorded utterance according to the determined at least one editing location, wherein the difference segments include the at least one edit segment.
11. A machine-readable program storage device tangibly embodying a program of instructions that, when executed by the machine, perform a method for generating synthesized speech from input text, the method comprising:
selecting a best-matched pre-recorded utterance from a plurality of pre-recorded utterances, wherein the selecting is based, at least in part, on a degree of matching between the input text and texts associated with the plurality of pre-recorded utterances;
dividing the best-matched pre-recorded utterance into a plurality of segments comprising remaining segments that match corresponding parts of the input text and difference segments that do not match corresponding parts of the input text;
synthesizing speech for parts of the input text corresponding to the difference segments in the selected best-matched pre-recorded utterance to generate synthesized speech segments; and
splicing the synthesized speech segments of the parts of the input text corresponding to the difference segments with the remaining segments of the selected best-matched pre-recorded utterance to generate the synthesized speech for the input text.
12. The device according to claim 11 , wherein selecting a best-matched pre-recorded utterance comprises:
calculating an edit-distance between the input text and each of the plurality of pre-recorded utterances;
selecting the pre-recorded utterance with a minimum edit-distance as the best-matched pre-recorded utterance; and
determining at least one edit operation for converting the best-matched pre-recorded utterance into the synthesized speech for the input text.
13. The device according to claim 12 , wherein calculating an edit-distance is performed as follows:
E
(
i
,
j
)
=
min
{
E
(
i
-
1
,
j
-
1
)
+
Dis
(
s
i
,
t
j
)
E
(
i
,
j
-
1
)
+
Del
(
t
j
)
E
(
i
-
1
,
j
)
+
Ins
(
s
i
)
}
where S=s 1 . . . s i . . . s N represents a sequence of words in the pre-recorded utterance, T=t 1 . . . t j . . . t M represents a sequence of words in the input text, E(i, j) represents an edit-distance for converting, s 1 . . . s i into t 1 . . . t j , Dis(s i , t j ) represents a substitution penalty when replacing word s i in the pre-recorded utterance with word t j in the input text, Ins(s i ) represents an insertion penalty for inserting s i and Del(t j ) represents a deletion penalty for deleting t j .
14. The device according to claim 12 , wherein determining at least one edit operation comprises:
determining at least one editing location and at least one corresponding editing type.
15. The device according to claim 14 , wherein dividing the best-matched pre-recorded utterance into a plurality of segments comprises:
according to the determined at least one editing location, chopping out at least one edit segment to be edited from the best-matched pre-recorded utterance, wherein the difference segments include the at least one edit segment.