Synthesizing speech from text
View Patent ↗Speech is synthesized for a given text by determining a sequence of phonetic components based on the text, determining a sequence of target phonetic elements associated phonetic components, determining a sequence of target event types associated with the phonetic components and determining a sequence of speech units from a plurality of stored speech unit candidates by use of a cost function. The cost function comprises a unit cost, a concatenation cost, and an event type cost for each speech unit in the sequence of speech units. The unit cost of a speech unit is determined with respect to the corresponding target phonetic element, while the concatenation cost of a speech unit is determined with respect to adjacent speech units and the event type cost of each speech unit is determined with respect to the corresponding target event type.
1. A method of synthesizing speech from text, the method comprising:
determining a sequence of phonetic components from the text;
determining a sequence of target phonetic elements from the determined sequence of phonetic components;
determining a sequence of target event types from the sequence of target phonetic components, wherein a target event type in the sequence of target event types represents a plurality of intonation realizations; and
selecting a plurality of speech units from a set of stored speech unit candidates to form a sequence of selected speech units, wherein a first speech unit in the sequence of selected speech units is selected using a cost function comprising a unit cost determined with respect to the target phonetic element corresponding to the first speech unit, a concatenation cost determined with respect to at least one speech unit adjacent to the first speech unit, and an event type cost determined with respect to a target event type corresponding to the first speech unit.
2. The method according to claim 1 , wherein each target event type is selected from a set of predetermined event types, wherein the set of predetermined event types was automatically derived from at least one annotated speech corpus.
3. The method according to claim 2 , wherein each target event type is associated with an event type description that provides a set of parameters for the target event type, said set of parameters specifying at least one of the following: a duration for the target event type, one or more changes of the fundamental frequency over the duration of the target event type, and an intensity variation over the duration of the target event type.
4. The method according to claim 3 , wherein the event type cost of a speech unit takes at least one of the following into account: a distance between the movement of the fundamental frequency of the speech unit and the movement of the fundamental frequency specified by the event type description of the corresponding event type and/or a distance between the intensity variation of the speech unit and the intensity variation specified by the event type description.
5. The method according to claim 3 , wherein each speech unit is associated with an event type and the event type cost of a speech unit takes at least one of the following into account: a distance between the event type of the corresponding phonetic component, the event type of the speech unit and of one or more preceding speech units, and the event type of one or more adjoining speech units, related to the corresponding phonetic component.
6. The method according to claim 4 , wherein said distance is evaluated using a perceptually-measured metric.
7. The method according to claim 3 , wherein the event type cost of a speech unit takes into account a size of an area defined with respect to movement of the fundamental frequency in the event type description of the corresponding event type and the movement of the fundamental frequency in one or more speech units that fall within the duration specified by the event type description.
8. The method according to claim 7 , wherein the distance is evaluated by use of a metric quantifying the distance between at least one of the following:
fundamental frequency movements of the speech unit, and intensity variations and a set of parameters of the corresponding target event type description.
9. The method of claim 2 , wherein the set of predetermined event types was derived automatically from the at least one annotated speech corpus at least in part by using a clustering procedure.
10. A computer program product for synthesizing speech from text, said computer program product comprising at least one non-transitory computer usable medium having computer usable program code embodied therewith, said computer usable program code comprising:
computer usable program code configured to determine a sequence of phonetic components from the text;
computer usable program code configured to determine a sequence of target phonetic elements from the determined sequence of phonetic components;
computer usable program code configured to determine a sequence of target event types from the sequence of target phonetic components, wherein a target event type in the sequence of target event types represents a plurality of intonation realizations;
computer usable program code configured to select a plurality of speech units from a set of stored speech unit candidates to form a sequence of selected speech units, wherein a first speech unit in the sequence of selected speech units is selected using a cost function comprising a unit cost determined with respect to a target phonetic element corresponding to the first speech unit, a concatenation cost determined with respect to at least one speech unit adjacent to the first unit, and an event type cost determined with respect to a target event type corresponding to the first speech unit.
11. The computer program product according to claim 10 , wherein said computer usable program code configured to determine a sequence of target event types from the sequence of target phonetic components further comprises computer usable program code configured to select each target event type from a set of predetermined event types, wherein the set of predetermined event types was automatically derived from at least one annotated speech corpus.
12. The computer program product according to claim 11 , wherein each target event type is associated with an event type description that provides a set of parameters for the target event type, said set of parameters specifying at least one of the following: a duration for the target event type, one or changes of the fundamental frequency over the duration of the target event type, and an intensity variation over the duration of the target event type.
13. The computer program product according to claim 12 further comprising:
computer usable program code configured to associate each speech unit with an event type; and
computer usable program code configured to determine the event type cost of a speech unit taking at least one of the following into account: a distance between the event type of the corresponding phonetic component, the event type of the speech unit and of one or more preceding speech units, and the event type of one or more adjoining speech units, related to the corresponding phonetic component.
14. The computer program product according to claim 13 further comprising computer usable program code configured to evaluate said distance using a perceptually-measured metric.
15. The computer program product according to claim 14 , wherein said computer-usable program code configured to determine the event type cost of a speech unit comprises computer-usable program code that takes at least one of the following into account: a distance between the movement of the fundamental frequency of the speech unit and the movement of the fundamental frequency specified by the event type description of the corresponding event type and/or a distance between the intensity variation of the speech unit and the intensity variation specified by the event type description.
16. The computer program product according to claim 13 wherein the computer usable program code configured to determine the event type cost of a speech unit further comprises computer usable program code configured to take into account a size of an area defined with respect to movement of the fundamental frequency in the event type description of the corresponding event type and the movement of the fundamental frequency in one or more speech units that fall within the duration specified by the event type description.
17. The computer program product according to claim 16 wherein said distance is evaluated by use of a metric quantifying the distance between at least one of the following: fundamental frequency movements of the speech unit, and intensity variations and a set of parameters of the corresponding target event type description.
18. The computer program product of claim 11 , wherein the set of predetermined event types was derived automatically from the at least one annotated speech corpus at least in part by using a clustering procedure.
19. An apparatus for synthesizing speech from text, the apparatus comprising:
a processor configured to perform a method comprising the acts of:
determining a sequence of phonetic components from the text;
determining a sequence of target phonetic elements from the determined sequence of phonetic components;
determining a sequence of target event types from the sequence of target phonetic components, wherein a target event type in the sequence of target event types represents a plurality of intonation realizations;
selecting a plurality of speech units from a set of stored speech unit candidates to form a sequence of selected speech units, wherein a first speech unit in the sequence of selected speech units is selected using a cost function comprising a unit cost determined with respect to a target phonetic element corresponding to the first speech unit, a concatenation cost determined with respect to at least one speech unit adjacent to the first speech unit, and an event type cost determined with respect to a target event type corresponding to the first speech unit.
20. The apparatus according to claim 19 wherein each target event type is selected from a set of predetermined event types, wherein the set of predetermined event types was automatically derived from at least one annotated speech corpus.
21. The apparatus according to claim 20 , wherein each target event type is associated with an event type description that provides a set of parameters for the target event type, said set of parameters specifying at least one of the following: a duration for the target event type, one or changes of the fundamental frequency over the duration of the target event type, and an intensity variation over the duration of the target event type.
22. The apparatus according to claim 21 , wherein the method comprises associating each speech unit with an event type and wherein the event type cost of a speech unit takes at least one of the following into account: a distance between the event type of the corresponding phonetic component, the event type of the speech unit and of one or more preceding speech units, and the event type of one or more adjoining speech units, related to the corresponding phonetic component.
23. The apparatus of claim 20 , wherein the set of predetermined event types was derived automatically from the at least one annotated speech corpus at least in part by using a clustering procedure.